Skip to content

code fix & added goldens in USCensusPEP_AgeSexRaceHispanicOrigin - #2134

Open
niveditasing wants to merge 2 commits into
datacommonsorg:masterfrom
niveditasing:fixed_hispanic
Open

code fix & added goldens in USCensusPEP_AgeSexRaceHispanicOrigin#2134
niveditasing wants to merge 2 commits into
datacommonsorg:masterfrom
niveditasing:fixed_hispanic

Conversation

@niveditasing

@niveditasing niveditasing commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Python Code Fixes (pep_asrh_download_script.py)

  • Bypassed Bot-Blocking: Added headers={"User-Agent": "Mozilla/5.0"} to both requests.head (during URL availability verification) and requests.get (during file downloads).
  • Added Polite Delays: Imported time and introduced a time.sleep(1) delay before running sequential HTTP requests, smoothing out traffic peaks and preventing server rate-limiting.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces validation configurations and golden data files for the US Census PEP ASRH import, including adding a validation_config.json and updating the manifest. Additionally, the download script pep_asrh_download_script.py was updated to include a polite delay and a custom User-Agent header when making requests. The review feedback correctly identifies that the time module is imported locally inside functions and loops, which violates PEP 8 guidelines, and recommends moving the import to the top of the file.

Comment thread statvar_imports/us_census/pep/us_census_pep_asrh/pep_asrh_download_script.py Outdated
Comment thread statvar_imports/us_census/pep/us_census_pep_asrh/pep_asrh_download_script.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant