govbot

Site Architecture

How both dashboards on this site are built — from raw government data, through automated pipelines, to the pages you're reading. Dashboard guide · GitHub repo

This site has three dashboards and three separate backend pipelines that feed them. They run automatically twice a day on GitHub Actions, write plain .json files into the site, and never break the page when a government source goes down — they fall back to the last good data. Here's how each one works. 👇

Legislation Dashboard data.json

Bills from 50+ states & territories. An AI model reads each bill and tags it by topic. Powers the searchable, filterable bill table and charts.

Committee Hearings hearings.json

Upcoming hearings & public-comment windows scraped straight from state legislatures and the federal register — with RSS feeds you can subscribe to.

Elections Happening in IL elections.json

Every office on upcoming Chicago & Illinois ballots — citywide, all 50 wards, CPS board, Police District Councils — with candidates from official candidate lists.

🗺️ The big picture

One static site, two data pipelines

Everything you see is static HTML + JSON hosted on GitHub Pages — no server, no database at request time. Two scheduled jobs refresh the data files behind the scenes; your browser just downloads them and draws the page.

Twice a day · GitHub Actions deploy-docs.yml

Legislation pipeline

clone 50+ state repos → AI topic-tag → data.json

Hearings pipeline

scrape legislatures + federal → hearings.json + RSS

Elections pipeline

official candidate lists → elections.json + RSS

Build the site

mdbook build

bundles the docs + copies the dashboard HTML & JSON verbatim

Deploy to GitHub Pages

publishes chihacknight.github.io/govbot

what you open in the browser

Legislation Dashboard index.html Hearings Dashboard hearings.html Elections Happening in IL elections.html This page architecture.html

50+

states & territories tracked

2×/day

rebuilt (08:00 & 20:00 UTC)

servers — fully static hosting

crashes when a source is down (fail-soft)

🏛️ Pipeline 1

The Legislation Dashboard

Git repositories are the dataset here: govbot mirrors each jurisdiction's bills as a git repo. The pipeline clones them, lets a small AI model read every new bill and label it by topic, then flattens the result into one data.json the page can search instantly.

Gather & tag GitHub Actions

govbot clone all

50+ legislation repos, shallow, 8 in parallel

Filter to new bills

filter_new_bills.py — a seen-bills ledger, so only fresh bills get re-tagged

govbot tag — AI topic tagging

all-MiniLM-L6-v2 embeddings vs. the topic taxonomy in govbot-dashboard.yml; keyword fallback if the model is unavailable

build_dashboard_data.py

merges every repo + tags into one data.json

Extra context fail-soft add-ons

Legislator roster → people.json

from the Open States people repo, so surname-only sponsors get full names

your browser draws it from data.json

Charts by state · topic · month Filters & search Sortable bill table Details modal live govbot record

Clone the data. Each state's bills live in their own git repo (git repos are the dataset). govbot clone all pulls them all down.
Skip what's unchanged. A ledger of already-seen bills means only new bills go through the expensive tagging step — the rest are reused.
Tag with a tiny AI model. An embedding model compares each bill's text to a curated list of topics and attaches the closest matches. If the model can't load, a keyword list stands in.
Flatten to one file. Everything is merged into data.json — the single file the dashboard downloads.
Render in the browser. index.html reads that file and builds the charts, filters, table, and per-bill details modal (which fetches the live govbot record on demand).

Fail-soft everywhere. Every network or AI step degrades to a warning plus a fallback (keyword tags, or the last committed sample data). A flaky government site never takes the dashboard down.

🗓️ Pipeline 2

Committee Hearings & Witness Slips

A completely separate scraper that taps legislature websites directly — not through Open States — plus federal public-comment periods. It produces a rolling near-future window of hearings and a family of RSS feeds so you can follow a single bill, a whole state, or one hearing.

Scrape the sources scrape-hearings/main.py

tapped directly — federal leads, then states

🇺🇸 USA Federal Regulations.gov Illinois ilga.gov Washington leg.wa.gov Massachusetts malegislature.gov Alaska akleg.gov

Parse & normalize

one shared schema (govbot.hearings.schema.json); federal falls back to federal_seed.json without an API key

outputs written into the site

hearings.json the dashboard data hearings.xml whole-calendar RSS per-bill RSS per-jurisdiction RSS per-hearing RSS participation.json 56-jurisdiction directory

hearings.html

renders the calendar, witness-slip windows, and RSS links

Go straight to the source. Instead of a third-party aggregator, the scraper reads each legislature's own site plus the federal Regulations.gov API.
Normalize to one shape. Every source is mapped onto a shared hearings schema so the page treats them all the same.
Fan out to feeds. Beyond the dashboard's hearings.json, it emits a whole-calendar RSS and granular per-bill / per-jurisdiction / per-hearing feeds you can subscribe to.
Keep the last good copy. If a source is down and zero hearings come back, the previous committed data stays — the panel never blanks out.

Tested offline. The parsers are snapshot-tested against saved copies of each site (python3 actions/scrape-hearings/test_scrape_hearings.py), so parsing works without hitting the network — exactly the offline-first principle the project is built on.

🗳️ Pipeline 3

Elections Happening in IL

The ballot's structure — every office, district, ballot date, and a plain-English “why this race exists” note — is stable, so it lives in a committed seed. Scrapers only populate candidates onto that seed from official candidate lists. A race with nobody listed yet keeps an empty slot and a link to its official source — never a guess.

Scrape candidate lists scrape-elections/main.py

official sources — no third-party aggregator

Chicago Board of Elections mayor · wards · PDC Illinois SBE “Who Is Running” · CPS board Cook County Clerk suburban (future)

Place onto the seed

elections_seed.json

96 races: citywide, wards 1–50, CPS president + subdistricts 1A–10B, 22 Police District Councils

Matcher

attaches a candidate to a race only when office + district resolve exactly; anything unplaceable is dropped, never invented

outputs written into the site

elections.json the dashboard data elections.xml whole-ballot RSS per-office-group RSS per-race RSS

elections.html

groups races by office, shows candidates + “why this race exists,” and RSS links

Start from the ballot's shape. A committed seed lists every race, its district, ballot date, and why it exists — this never depends on a scrape succeeding.
Read official candidate lists. Best-effort parsers read the Chicago Board of Elections pages and the Illinois SBE “Who Is Running” list.
Match, don't invent. Each candidate attaches to a race only when its office and district resolve unambiguously; unplaceable rows are dropped with a warning.
Fan out to feeds. Beyond elections.json, it emits a whole-ballot RSS plus per-office-group and per-race feeds you can follow.

Why the CPS races are special. On Nov 3, 2026 Chicago elects a school board by popular vote for the first time — including its first-ever elected president — as the mayor-appointed hybrid board is phased out. The seed carries that context on every CPS race so the page can explain it, not just list it.

🔗 Where they meet

Pipelines that talk to each other

The three dashboards are independent, but they cross-reference each other so a bill and its hearing feel connected.

🏛️ A bill's details modal

Opens a bill on the Legislation Dashboard and checks the hearings feed…

🗓️ …shows if it's on a calendar

“On the committee calendar — see the Committee Hearings tab.” And hearings link bills back with a pre-filtered deep link.

🧰 Under the hood

What it's built with

Deliberately boring, portable, and dependency-light — everything runs as plain scripts.

Rust — the govbot CLI Python — build & scrape scripts all-MiniLM-L6-v2 — embeddings GitHub Actions — scheduler mdBook — site builder GitHub Pages — hosting Git repos — the datasets Vanilla JS — no framework JSON Schema — validation

Want the deeper walkthrough? The dashboard guide documents the data flow, and every action folder in the GitHub repo is a self-contained module you can run as a shell script or a GitHub Action.

A Chi Hack Night project