feat: document Index file in architecture.md

This commit is contained in:
shenleban tongying 2024-03-23 00:05:13 -04:00
parent e0d09302d2
commit 49e576d02f
No known key found for this signature in database
3 changed files with 45 additions and 23 deletions

View file

@ -0,0 +1,30 @@
## Index file
Each index file have 4 sections.
1. `IdxHeader`
2. `ExtraInfo` (Being used but unnamed in source code)
3. `Chunks`
4. `BtreeIndex`
The `IdxHeader` are 32bits blocks of various meta info of the index. The most important info are `chunksOffset` and `indexRootOffset` pointing to the starting offset of `BtreeIndex` and `Chunks`.
Some dicts only have one `ExtraInfo`: the `dictionaryName` which is an uint32 size of a string followed and the string.
Each chunk contains uint32 size of uncompressed data, uint32 size of zlib compressed data, and the zlib compressed data.
The `Chunks` maybe used by both `IdxHeader` and `BtreeIndex`.
By adding new a new chunk to `Chunks` and store an offset to `IdxHeader`, `ExtraInfo` can store arbitrary long information.
`BtreeIndex` is a zlib compressed typical btree implementation in which each Node will include `word` info and a `offset` that pointing to corresponding `chunk`'s position.
Note that a `chunk` only includes necessary data to find an article, and it does not contain the `word`.
The exact data in `chunk` is decided and interpreted by dictionary implementations. For example, the starting and ending position of an article in a dictionary file.
## What's under the hood after a word is queried?
After typing a word into the search box and press enter, the embedded browser will load `gdlookup://localhost?word=<wantted word>`. This url will be handled by Qt webengine's Url Scheme handler. The returned html page will be composed in the ArticleMaker which will initiate some DataRequest on dictionary formats. Resource files will be requested via `bres://` or `qrc://` which will went through a similar process.
TODO: other subsystems.

View file

@ -16,12 +16,4 @@ Commit messages should follow [Conventional Commits](https://www.conventionalcom
Reformat changes with `clang-format` [how to use clang-format](https://github.com/xiaoyifang/goldendict/blob/staged/howto/how%20to%20use%20.clang-format%20to%20format%20the%20code.md) Reformat changes with `clang-format` [how to use clang-format](https://github.com/xiaoyifang/goldendict/blob/staged/howto/how%20to%20use%20.clang-format%20to%20format%20the%20code.md)
Remember to enable `clang-tidy` support on your editor so that `.clang-tidy` will be respected. Remember to enable `clang-tidy` support on your editor so that `.clang-tidy` will be respected.
## Architecture
What's under the hood after a word is queried?
After typing a word into the search box and press enter, the embedded browser will load `gdlookup://localhost?word=<wantted word>`. This url will be handled by Qt webengine's Url Scheme handler. The returned html page will be composed in the ArticleMaker which will initiate some DataRequest on dictionary formats. Resource files will be requested via `bres://` or `qrc://` which will went through a similar process.
TODO: other subsystems.

View file

@ -1,4 +1,4 @@
site_name: GoldenDict-NG site_name: GoldenDict-ng
site_description: GoldenDict-ng is a open source, cross platform, multi formats, feature rich dictionary 是一个开源跨平台支持各种格式的字典程序 site_description: GoldenDict-ng is a open source, cross platform, multi formats, feature rich dictionary 是一个开源跨平台支持各种格式的字典程序
site_url: https://xiaoyifang.github.io/goldendict-ng/ site_url: https://xiaoyifang.github.io/goldendict-ng/
@ -36,8 +36,9 @@ nav:
- ToolBar & DictBar: ui_toolbar.md - ToolBar & DictBar: ui_toolbar.md
- Favorites: ui_favorites.md - Favorites: ui_favorites.md
- Shortcuts: ui_shortcuts.md - Shortcuts: ui_shortcuts.md
- Special Usages: - Advanced Usages:
- Anki Integration: topic_anki.md - Anki Integration: topic_anki.md
- Program dictionary: howto/how to add a program as dictionary.md
- Command Lines: topic_commandline.md - Command Lines: topic_commandline.md
- Custom Stylesheet & JavaScript: topic_userstyle.md - Custom Stylesheet & JavaScript: topic_userstyle.md
- Portable Mode: topic_portablemode.md - Portable Mode: topic_portablemode.md
@ -45,17 +46,16 @@ nav:
- Customize Dictionary: custom_dictionary.md - Customize Dictionary: custom_dictionary.md
- OCR Integration: howto/ocr.md - OCR Integration: howto/ocr.md
- Wayland: topic_wayland.md - Wayland: topic_wayland.md
- Debug dictionary JS: howto/how to debug dictionary js.md
- Report Bugs & Feedbacks: feedbacks.md - Report Bugs & Feedbacks: feedbacks.md
- Contributor Guides: - Development Info:
- Developer: developer.md - Start develop: developer.md
- How to: - Architecture: architecture.md
- Build from source: howto/build_from_source.md - Build from source: howto/build_from_source.md
- Customize the opencc: howto/how to customize the opencc.md - Customize the opencc: howto/how to customize the opencc.md
- Qt version and github action: howto/how to find out the latest qt version and module in github qt action.md - Qt version and github action: howto/how to find out the latest qt version and module in github qt action.md
- Use .clang-format: howto/how to use .clang-format to format the code.md - Use .clang-format: howto/how to use .clang-format to format the code.md
- Breadpad crash analysis: howto/how to use breadpad crash analysis.md - Breadpad crash analysis: howto/how to use breadpad crash analysis.md
- Build ffmpeg on Windows: howto/how to build ffmpeg for visual studio.md - Build ffmpeg on Windows: howto/how to build ffmpeg for visual studio.md
- How to update the crowdin.ts file: howto/how to update crowdin.ts file.md - Update the crowdin.ts file: howto/how to update crowdin.ts file.md
- How to debug dictionary js: howto/how to debug dictionary js.md
- How to add a program dictionary: howto/how to add a program as dictionary.md