index
:
crawler
main
Asynchronous webcrawler in Python.
Brandon Irizarry
summary
refs
log
tree
commit
diff
log msg
author
committer
range
Age
Commit message (
Collapse
)
Author
3 days
chore: untrack description
Brandon C. Irizarry
3 days
chore: add description for cgit
Brandon C. Irizarry
3 days
feat: configure max_concurrency from the command line
Brandon C. Irizarry
3 days
docs: add docstring to 'fetch'
Brandon C. Irizarry
3 days
feat: use a semaphore to limit how many tasks make requests
Brandon C. Irizarry
3 days
feat: use taskgroup + flush printing of '.' + catch timeouterror
Brandon C. Irizarry
3 days
feat: use full concurrency (no semaphores)
Brandon C. Irizarry
3 days
feat: add a timeout for sites that don't respond
Brandon C. Irizarry
This should diminish the urge for reaching for Ctrl+C.
3 days
feat: ping sites sequentially and measure how long it takes
Brandon C. Irizarry
3 days
docs: improve docstring
Brandon C. Irizarry
That is, make it more accurate and specific.
3 days
chore: install aiohttp
Brandon C. Irizarry
3 days
feat: read N URLs via command-line parameter from CSV file
Brandon C. Irizarry
3 days
chore: start project
Brandon C. Irizarry