Ignite IT Hub is seeking a Senior Data Scientist to improve the U.S. Census Bureau's LLM Autocoder and assess its use across household surveys and demographic projects. You will lead analytical methods and experimental design for coding survey write-in responses, working with engineers, the Model Evaluation Analyst, and Government subject matter experts (SMEs) to turn evidence into practical recommendations.
Responsibilities
- Establish performance baselines and evaluation methods for the American Community Survey (ACS) Autocoder and proposed expansion efforts.
- Analyze accuracy, industry and occupation joint coding rates, coverage, error patterns, and category-level results to identify improvement opportunities and regressions.
- Design reproducible experiments with representative samples, appropriate comparison groups, and controls against data leakage; explain uncertainty and limitations.
- Compare LLMs, fuzzy matching, direct-match dictionaries, rules, and hybrid approaches based on coding quality, explainability, feasibility, and operational cost.
- Assess data readiness and coding taxonomies for additional surveys and ACS fields, such as race and ancestry, field of degree, and language.
- Work with Government SMEs to resolve ambiguous cases and refine evaluation criteria; guide engineers on changes to prompts, preprocessing, model configuration, and coding methods.
- Interpret pilot and production results, recommend whether to implement or revise a method, and document findings, reusable practices, and technical guidance.
Requirements
Required qualifications
- Bachelor's degree in data science, statistics, computer science, mathematics, or a related field, or equivalent relevant professional experience.
- 8+ years in applied data science or statistical analysis, including 3+ years with NLP or text classification and hands-on LLM application experience.
- Strong Python and SQL skills, with experience preparing and analyzing large datasets and using statistical and machine learning libraries.
- Strong experimental design and model evaluation skills, including sampling, class imbalance, error analysis, and reliable comparisons against baselines.
- Experience supporting deployed models, collaborating with engineers, and explaining analytical findings and limitations to technical and nontechnical audiences.
Preferred qualifications
- Advanced degree in a relevant field; experience with survey data, industry and occupation coding, or other detailed classification taxonomies.
- Experience with AWS-based analytical workflows and federal statistical or sensitive-data environments.
Conditions of employment. Position contingent on contract award. Must be able to obtain and maintain applicable Census Special Sworn Status and Personal Identity Verification (PIV) credentials, follow confidentiality requirements, and support Census core business hours and coordinated critical processing runs.
Benefits
Benefits
- 401(k)
- 401(k) matching
- Dental insurance
- Flexible spending account
- Health insurance
- Life insurance
- Paid time off
- Professional development assistance
- Referral program
- Tuition reimbursement
- Vision insurance