hashirama/jitenbot
1
0
Fork 0
Code Issues Pull requests Actions Projects Releases Wiki Activity

Update README.md

Browse source
This commit is contained in:
Stephen Kraus 2023-05-01 23:53:22 -05:00 committed by GitHub
parent 2bd6ea1322
commit e6c892967a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
README.md
View file

@ -64,7 +64,7 @@ options:
### Online Targets ### Online Targets
Jitenbot will scrape the target website and save the pages to the [user cache directory](https://pypi.org/project/platformdirs/). Jitenbot will scrape the target website and save the pages to the [user cache directory](https://pypi.org/project/platformdirs/).
As a courtesy to the website owners, jitenbot is configured to pause for 10 seconds between each page request. Consequently, As a courtesy to the website owners, jitenbot is configured to pause for 10 seconds between each page request. Consequently,
a complete crawl of a target website may take several hours. a complete crawl of a target website may take several days.
HTTP request headers (user agent string, etc.) may be customized by editing the `config.json` file created in the HTTP request headers (user agent string, etc.) may be customized by editing the `config.json` file created in the
[user config directory](https://pypi.org/project/platformdirs/). [user config directory](https://pypi.org/project/platformdirs/).