zlacker

This is super important work, and is kind of why I built https://civic.band and https://civic.observer, which are generalized tools for monitoring civic govts. (You can search for anything, not just ALPR)

replies(4): >>ZeWaka+48 >>kpw94+7r >>tayari+Km2 >>tonyme+NM3

>>phildi+(OP)
Seem to be getting 405s from https://civic.observer/auth/login

And 404 from https://civic.band/why.html

replies(1): >>phildi+qd

>>ZeWaka+48
Fixed the 404s on civic.band, thanks

>>phildi+(OP)
Very cool! And important for sure, thank you.

Few questions:

- is the stack to index those open source?

- is there some standardized APIs each municipality provides, or do you go through the tedious task of building a per-municipality crawling tool?

- how often do you refresh the data? Checked a city, it has meeting minutes until 6/17, but the official website has more recent minutes (up to 12/2 at least)

replies(1): >>phildi+tB

>>kpw94+7r
Thanks for asking!

- The framework for crawling is open-source. https://github.com/civicband

- There is absolutely not a standardized API for nearly any of this. I build generalized crawlers when I can, and then build custom crawlers when I need.

- Can you let me know which city? The crawlers run for every municipality at least once every day, so that's probably a bug

>>phildi+(OP)
This is incredible, great work and will definitely be using and sharing this!

Where in the repos can we find the plugin/scraper for given municipalities to help contribute when they seem to be broken? As looks like the last meetings and agendas scraped for Cook County are from March/April of this year

replies(2): >>mdnaha+xo2 >>phildi+YF3

>>tayari+Km2
Also missing: Austin, TX

replies(1): >>phildi+0G3

>>tayari+Km2
Hello! The crawlers are not currently public, but I'm happy to take volunteers behind the curtain.