urls - Web crawler in Go.

Age	Commit message (Collapse)	Author
2026-06-11	feat: write XML sitemaps to separate directory	demo

2026-05-28	feat: implement shortcode feature	demo

2026-05-28	feat: add comment logging maxDepth and maxURLs inside xml output	demo

2026-05-28	feat: save sitemap to a file	demo

2026-05-28	wip: generate rough draft of sitemap	demo

2026-05-28	refactor: move html document creation to getBatch	demo
	Also, if there are errors, I log them and simply return a nil slice.
2026-05-27	fix: make select statement block unless communication takes place	demo

2026-05-27	feat: configure maxDepth from the command line	demo
	Similar to maxURLs, a maxDepth of zero means no limit.
2026-05-27	wip: prototype a max-depth limitation	demo
	It's just bolted on with a constant right now though.
2026-05-27	feat: update the classic crawler to track depth via packets	demo

2026-05-26	fix: release semaphore at the proper time	demo

2026-05-26	feat: implement maxConcurrency using a buffered channel 'sema'	demo

2026-05-26	feat: add cancellation feature	demo

2026-05-26	feat: hit 'em with the classic web crawler	demo