Devon Canora

Job Title: AI Lead We are partnering with a leading global organization to identify an AI Lead - Platform Intelligence & Applied AI to join its enterprise delivery team. This role is primarily remote, with occasional on-site meetings as needed. Candidates must be comfortable working Eastern Time business hours.

This is a full-time contract engagement (40 hours per week) open to credentialed professionals located within the United States. Candidates should be prepared to complete standard background screening and verification processes in accordance with applicable laws and company policies.

Role Overview

We are seeking an AI Lead to serve as the technical authority and strategic driver for how artificial intelligence is designed, implemented, and continuously evolved within our enterprise advisory delivery platform.

This role combines deep technical expertise with strategic vision, ensuring the platform leverages modern AI capabilities in a scalable, reliable, and enterprise-ready manner. The AI Lead will maintain a hands-on understanding of emerging AI technologies while translating research and market advancements into practical solutions that enhance the platform’s intelligence and capabilities.

You will help define how models are used, how intelligence is orchestrated, how contextual data is assembled, how agents operate, and how AI performance, trust, and quality are measured at scale. Key Responsibilities AI Strategy & Market Intelligence

• Continuously evaluate developments in:

  • Large language models (LLMs) and foundation models

  • Agent frameworks and orchestration architectures

  • Retrieval, memory, and contextual intelligence techniques

  • AI evaluation, safety, and governance frameworks

• Translate emerging AI capabilities into:

  • Platform design principles

  • Proofs of concept and experimental initiatives

  • Scalable, production-ready capabilities

• Provide strategic guidance to leadership on the adoption and integration of new AI technologies.

Model & Intelligence Management

• Define and manage the strategy for model usage across the platform, including:

  • Model selection and benchmarking

  • Versioning and lifecycle management

  • Performance, cost, and latency optimization

  • Redundancy and fallback strategies

  • Abstraction layers supporting multi-vendor model integration

• Establish best practices for:

  • Prompt and instruction design

  • Tool and function calling

  • Structured outputs and deterministic system behavior.

Semantic Routing & Orchestration

• Design and evolve the platform’s semantic routing layer, including:

  • Intent detection and task classification

  • Intelligent routing to appropriate models, agents, or workflows

  • Context-aware decision-making based on workspace state

• Define orchestration patterns for:

  • Multi-step and parallel execution

  • Long-running and asynchronous tasks

  • Human-in-the-loop workflows

• Ensure routing logic is transparent, testable, and continuously optimized.

Agent Architecture & Execution

• Define the organization’s AI agent strategy, including:

  • When agents should be used versus workflows or direct model calls

  • Agent composition, memory design, and tool integration

  • Guardrails and behavioral constraints

• Partner with engineering teams to implement:

  • Agent frameworks and runtime infrastructure

  • Monitoring, debugging, and performance management capabilities

• Ensure agents are:

  • Predictable, auditable, and secure

  • Aligned with delivery methodologies and service workflows

  • Safe for enterprise and client-facing environments.

Workspace Context & Retrieval Architecture

• Own the design of contextual intelligence and retrieval architecture, including:

  • Document ingestion, chunking, and enrichment pipelines

  • Vector, keyword, and hybrid retrieval approaches

  • Context assembly across client data, internal knowledge, and engagement artifacts

• Define standards for:

  • Source attribution and transparency

  • Data isolation, privacy, and compliance

  • Relevance, freshness, and system performance

• Continuously evaluate emerging approaches to memory, retrieval, and grounding.

AI Evaluation, Testing & Trust

• Establish and maintain the platform’s AI evaluation and testing framework, including:

  • Scenario-based and task-based evaluation methodologies

  • Regression testing for prompts, agents, and routing logic

  • Comparative benchmarking across models and configurations

• Define performance metrics for:

  • Accuracy, relevance, and consistency

  • Cost efficiency and latency

  • User trust, explainability, and reliability

• Collaborate with engineering and risk teams to ensure:

  • Full observability into AI system behavior

  • Safe deployment processes and controlled experimentation.

Qualifications

• 5+ years of experience working with modern AI systems, machine learning platforms, or intelligent application architectures.

• Strong understanding of large language models, agent architectures, and retrieval-augmented generation (RAG) frameworks.

• Experience designing and implementing AI-driven enterprise platforms or intelligent systems.

• Demonstrated ability to translate emerging AI technologies into scalable production solutions.

• Excellent collaboration and communication skills, with the ability to work effectively with engineering, product, and business stakeholders.

• Bachelor’s degree required in Computer Science, Engineering, Artificial Intelligence, or a related technical field.

By applying for this job, you agree to receive calls, AI-generated calls, text messages, or emails from Benchmark IT, LLC and its affiliates, and contracted partners. Frequency varies for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You can reply STOP to cancel and HELP for help. You can access our privacy policy here: https://bmarkits.com/privacy-policy/