urls, branch master Web crawler in Go. https://git.brandonirizarry.xyz/urls/atom?h=master 2026-06-12T17:10:19Z docs: use proper Markdown syntax 2026-06-12T17:10:19Z demo demo@antix1 2026-06-12T17:10:19Z urn:sha1:ea9dfe23d76dad076beace0b8ea728022d33d1ff feat: guard against non-HTML content types 2026-06-12T16:32:06Z demo demo@antix1 2026-06-12T16:32:06Z urn:sha1:597d4de8dcdc7d0f9d906a0e99be3d72ed43b1cb chore: add crawler-test.com to shortcodes as 'crawlertest' 2026-06-12T16:16:40Z demo demo@antix1 2026-06-12T16:16:40Z urn:sha1:4dcfd3a5378f8ba6edfc19dff76813d88ad199b2 refactor: return errors instead of using log.Fatal 2026-06-12T15:48:53Z demo demo@antix1 2026-06-12T15:48:53Z urn:sha1:d63791257350c7cafe792d4ebfc625792e81ab65 refactor: move sitemap-creation to 'run' and fix missing xml dir 2026-06-12T15:45:59Z demo demo@antix1 2026-06-12T15:45:59Z urn:sha1:104577455dcc0d45d0c0b6b9f07221beb1eb455f feat: use getLeakProfile to check for goroutine leaks in 'classic' 2026-06-12T15:32:03Z demo demo@antix1 2026-06-12T15:32:03Z urn:sha1:3d1aa712102072cd2b0c44ff007d06ac432909d6 refactor: move main business logic into run.go 2026-06-12T15:30:40Z demo demo@antix1 2026-06-12T15:30:40Z urn:sha1:ed605f2643b369a14042527209ceef16e43bbc44 refactor: move Action code to separate function 2026-06-12T15:26:53Z demo demo@antix1 2026-06-12T15:26:53Z urn:sha1:1e8f0e20ee895394209f070b05137b1bec609534 refactor: remove else-branch when deciding which flag to use 2026-06-12T15:20:22Z demo demo@antix1 2026-06-12T15:20:22Z urn:sha1:89ea120d7a8e001653160dffcb35f19fef6c7bfe refactor: streamline action-logic calling the "classic" crawler 2026-06-12T15:15:46Z demo demo@antix1 2026-06-12T15:15:46Z urn:sha1:317d4ba474733482492dc44385f4321456fde3e8