About the Role
Lead the development of agent intelligence that helps mechanical engineers complete multi-step workflows in desktop engineering software. You will combine hands-on engineering, technical leadership, and collaboration with user research and domain experts to improve real-world task completion and product value.
What You'll Do
Own agent task success metrics, establish baselines, and systematically improve completion rates across engineering workflows.
Design and build reproducible evaluation infrastructure based on validated user stories, including tests for cost efficiency and failure modes.
Set per-task token budgets and track the cost of completed workflows.
Interview engineers with research and domain teammates, document their workflows, and translate validated stories into benchmarks.
Prioritize workflow coverage based on customer value and technical feasibility.
Set the agent architecture for tool calling, state management, error recovery, model routing, and context management.
Lead a small AI engineering team while writing production code, reviewing designs, and unblocking teammates.
Work with product, integrations, and customers during proof-of-concept projects to align agent behavior with real usage.
What We're Looking For
At least 7 years of software engineering experience, including 2 or more years building LLM-based agents that take real-world actions.
Strong Python skills and deep experience with LLM application architecture, including model selection, retrieval, context management, orchestration, and tool calling.
Experience building rigorous agent evaluation and benchmarking systems that measure task completion, cost, and failure modes.
Technical leadership experience on a small engineering team, with a track record of reviewing code and continuing to contribute production software.
Experience shipping AI or LLM tools using proprietary engineering data or desktop engineering software.
Familiarity with function calling, tool APIs, tracing or observability, and evaluation tools.
Experience with desktop automation or application APIs, such as COM, and a background in mechanical engineering software, CAD/CAE, PLM, or an adjacent domain.
Understanding of deployment constraints on managed enterprise workstations.
Compensation & Benefits
Base salary: $160,000 to $250,000 USD annually. Visa sponsorship is not available.
Location
On-site in San Francisco, California, United States.