evaluation-frameworks jobs in San Diego

Forward Deployed Engineer who embeds with clients to identify business needs and translate high-value GenAI use cases into production solutions. Build AI-enabled solutions, agentic platforms, and workflows across enterprise AI platforms, delivering production-quality code with strong testing, CI/CD, logging, and documentation practices.…See more Lead working sessions, prototype solutions, and mentor team members while balancing architecture decisions around quality, safety, latency, cost, and model risk. Requires 3+ years software/data engineering experience, 1+ years hands-on GenAI/LLM deployment, and 1+ years with Microsoft AI & Data (Azure AI Foundry).

A forward-deployed engineer who builds and deploys AI/GenAI-powered solutions on Databricks and cloud platforms for forensic discovery and financial crime clients. The role requires hands-on expertise in Databricks, cloud services (AWS/Azure/GCP), and production AI/LLM systems, combined with the ability to lead client engagements, translate business requirements into technical architectures, and troubleshoot integration issues across enterprise environments.…See more You will design reusable assets, document workflows, and enable client teams to sustain delivered solutions while navigating security, compliance, and privacy reviews.

Develop scalable AI engineering patterns and production-quality code for government clients, balancing quality, safety, latency, cost, and model risk. Apply strong practices in testing, CI/CD, logging, versioning, and documentation while designing extensible functionality aligned with senior team members.…See more Requires 4+ years of software/data engineering experience, 1+ year building and deploying GenAI or LLM solutions, and 1+ year with Google Cloud (Gemini API, Vertex AI Agent Builder, Grounding, or Google Workspace integration). Lead project workstreams, translate business problems into AI solutions, and contribute reusable assets including code, prompt libraries, runbooks, and reference implementations.

Sr. Engineer responsible for designing, building, and deploying production-grade AI platforms and solutions centered on LLMs, RAG, and agentic workflows.…See more The role spans full-stack development—from backend services and APIs to cloud-native infrastructure on Kubernetes—with emphasis on enterprise search, retrieval optimization, and AI application integration. Must have strong programming skills across Python, Java, C#/.NET, Rust, TypeScript/JavaScript, and proven experience architecting and operating scalable, production-ready AI systems.

Staff Engineer or Architect role focused on building and orchestrating AI agents in a compliance-heavy, globally regulated environment. You will troubleshoot novel failures across models, tools, and infrastructure; set technical vision; and leverage LLMs, RAG, tool use, and agent frameworks (LangGraph, CrewAI, AutoGen) in production systems.…See more Requires 8+ years of software engineering experience, hands-on LLM and agent-framework work, end-to-end ownership, and demonstrated ability to use AI agents effectively in your own workflow while maintaining accountability and clear mental models of system behavior.

Lead a team of Databricks engineers delivering GenAI and LLM solutions to government clients. You'll architect and deploy production-scale data platforms, mentor engineers, and translate business requirements into AI solutions using Databricks' full stack (Lakehouse, Agent Bricks, Model Serving, Genie, Apps).…See more Enforce strong data management, CI/CD, testing, and documentation practices while working across AWS, Azure, and Google Cloud environments.

Senior Forward Deployed Engineer at Deloitte GPS building AI-enabled solutions and agentic platforms on Databricks for enterprise government clients. Requires 7+ years software/data engineering experience, 5+ years deploying GenAI/LLM solutions in production, and 5+ years hands-on Databricks expertise across Lakehouse, Agent Bricks, Model Serving, and Genie.…See more Mentors junior engineers, designs scalable AI patterns with human-in-the-loop controls, delivers production-quality code with strong CI/CD and testing practices, and translates complex business problems into deployable AI solutions.

Senior engineer at Deloitte GPS delivering production-quality code for GenAI/LLM-powered solutions, with 7+ years in software/data engineering and 1+ years hands-on experience building and deploying Claude-based applications. Responsibilities include designing extensible functionality, leading project workstreams, translating business problems into AI solutions, and contributing reusable assets (code, prompt libraries, runbooks).…See more Strong emphasis on testing, CI/CD, logging, documentation, and mentoring junior team members in fast-paced, ambiguous client delivery environments.

Senior Forward Deployed Engineer responsible for building and deploying production-grade AI solutions using frontier GenAI platforms (Claude, Gemini, GPT-4o) in client environments. Will lead end-to-end solution delivery, including architecture decisions, demo/POC development, and client enablement while mentoring junior engineers.…See more Requires 5+ years software/data engineering experience, 1+ years hands-on GenAI/LLM development, and demonstrated ability to translate business problems into scalable AI workflows with strong engineering practices (CI/CD, testing, documentation).

Principal Engineer leading the technical direction for ServiceNow's Conversational Analytics platform, which enables customers to query data in natural language using AI. The role combines hands-on architecture and prototyping with mentorship of engineering teams, requiring deep expertise in AI/ML systems, vector retrieval, agentic architectures, and knowledge graphs.…See more You'll collaborate with product and leadership to design cloud-based solutions integrating LLMs and AI into enterprise workflows, with accountability for customer success metrics and evaluation frameworks.

Senior Forward Deployed Engineer responsible for building and deploying GenAI/LLM-powered solutions on AWS, working directly with enterprise clients to translate business needs into production AI systems. The role requires 5+ years of software/data engineering experience, 1+ years hands-on with AWS AI&Data services (Bedrock, Neptune, OpenSearch), and the ability to lead technical workstreams while mentoring team members.…See more You will prototype solutions, develop scalable AI patterns with human-in-the-loop controls, write production-quality code with strong testing and CI/CD practices, and create reusable assets including code libraries and reference implementations. 50% travel required.

Forward Deployed Engineer who partners with enterprise clients to identify business needs and prototype production-grade GenAI/LLM solutions on AWS. Designs and builds AI-enabled agentic platforms, workflows, and scalable engineering patterns using AWS AI&Data services (Bedrock, Neptune, OpenSearch).…See more Delivers production-quality code with strong testing, CI/CD, logging, and documentation practices while mentoring team members and translating complex business problems into technical AI solutions.

Senior Microsoft Forward Deployed Engineer who will design, build, and deploy GenAI/LLM-powered solutions for government clients, translating business problems into production-grade AI systems. The role requires 7+ years of software/data engineering experience, 1+ years hands-on with GenAI/LLM solutions, and deep expertise in Microsoft Azure (AI Foundry, OpenAI, AI Search), Python/TypeScript/C#, and Copilot extensibility.…See more You will lead project workstreams, work directly with client technical teams in fast-paced environments, and mentor others while building reliable, maintainable, well-documented code. 50% travel required.

Build the foundational agentic AI layer for a materials-science platform, including multi-model provider abstraction, agent orchestration with stateful checkpoints, retrieval systems, prompt versioning, and comprehensive tracing and evaluation frameworks. You'll design agents that plan and reason over tool calls in production, implement human-in-the-loop safety gates, and ensure all LLM behavior remains auditable and cost-tracked across customers' secure environments.…See more The role demands deep production experience with agentic and LLM systems: async Python, structured outputs, memory and context management, multi-step workflow orchestration, and evaluation harnesses that catch regressions before deployment.