assessment-tool
AI Fluency Assessment Rubric
Role-based rubric for assessing organizational AI fluency across functions. Rates individuals and teams from Unacceptable to Transformative across Engineering, Product, Support, People/HR, and Marketing. Used during Audit phase to baseline human capital readiness for AI transformation.
Purpose
This rubric assesses individual and team AI fluency across key organizational functions. It provides:
- Baseline measurement during Audit phase
- Gap identification for training/hiring priorities
- Progress tracking through transformation
- Redeployment candidate identification for crystallized capacity
Key Insight: AI transformation success correlates strongly with workforce AI fluency. Per BCG research, organizations that train employees on AI are 2x more likely to achieve transformation outcomes.
Section 1: The Fluency Rubric
Maturity Levels
| Level | Score | Description |
|---|---|---|
| Unacceptable | 1 | Resistance or ignorance; actively avoiding AI; no experimentation |
| Capable | 2 | Basic usage for personal productivity; understands fundamentals |
| Adoptive | 3 | Integrated into workflows; measures impact; iterates on approaches |
| Transformative | 4 | Drives org-wide AI strategy; ships AI-powered products/features; thought leader |
1.1 Engineering
| Level | Indicators |
|---|---|
| Unacceptable (1) | β’ Calls AI coding assistants "too risky" |
| β’ Has never tested AI-generated code | |
| β’ Relies only on Stack Overflow snippets | |
| Capable (2) | β’ Uses ChatGPT/Copilot/etc for simple coding tasks (regex, unit-test stubs) |
| β’ Can explain how they prompt, review, and validate AI output | |
| Adoptive (3) | β’ Chains LLM calls with fallback + retry logic |
| β’ Adds eval tests to flag hallucinations | |
| β’ Knows Claude Code, Cursor, Windsurf, etc. | |
| β’ Can walk interviewers through prompt tweaks, token limits, code-review, etc. | |
| Transformative (4) | β’ Ships LLM-powered features, monitors live metrics, and refines based on user feedback |
| β’ Builds an AI-first dev pipeline (guardrails, RAG docs, etc.) that cuts down PR cycle time |
Assessment Questions:
- What AI coding tools do you use daily? How do you validate their output?
- Describe a time you debugged an AI-generated code issue.
- How would you architect an LLM-powered feature with reliability requirements?
1.2 Product
| Level | Indicators |
|---|---|
| Unacceptable (1) | β’ Dismisses AI as hype, showing no curiosity about user value |
| β’ PRDs and prototypes lack any AI concepts or experiments | |
| Capable (2) | β’ Uses ChatGPT to draft PRDs, story maps, and synthesize user-interview notes |
| β’ Knows basics (LLMs, embeddings, latency vs. cost) and can share example prompt patterns | |
| Adoptive (3) | β’ Ships an AI-powered feature with a clear "human-in-the-loop" check |
| β’ Chooses models based on accuracy, latency, throughput, and context-window constraints | |
| β’ Demonstrates ROI (e.g., cut time-to-insight from 5 days down to 3) | |
| Transformative (4) | β’ Builds/drives product strategy & org-wide AI roadmap through eval-first product development |
| β’ Launches a proprietary fine-tuned LLM feature that opens up a new pricing tier |
Assessment Questions:
- How do you evaluate whether a feature should use AI vs. traditional logic?
- Walk me through an AI feature you shippedβhow did you measure success?
- How do you think about AI's impact on your product's competitive moat?
1.3 Support
| Level | Indicators |
|---|---|
| Unacceptable (1) | β’ Refuses to use AI for support workflows |
| β’ Has zero automation skills (no rules, macros, bots, etc.) | |
| β’ Manually handles every ticket | |
| Capable (2) | β’ Summarizes tickets with ChatGPT and cites faster context shifts |
| β’ Knows and follows Security / TSO approval flow before trying new tools | |
| Adoptive (3) | β’ Builds Zapier workflows that triage queues and auto-tag CRM records |
| β’ Tracks CX metrics, refines prompts when AI misreads tone, keeps living doc of what works | |
| Transformative (4) | β’ Rolls out an org-wide AI triage bot that cuts first-response time by 25% |
| β’ Creates & presents ROI dashboards, balancing cost vs. customer experience in quarterly planning |
Assessment Questions:
- How do you currently use AI in your support workflow?
- Describe a time AI misunderstood customer intentβhow did you handle it?
- How would you measure the ROI of an AI support tool?
1.4 People / HR
| Level | Indicators |
|---|---|
| Unacceptable (1) | β’ Distrusts all AI hiring tools |
| β’ Screens each resume one-by-one | |
| β’ Relies on manual scheduling and candidate follow-ups | |
| Capable (2) | β’ Drafts interview guides & summarizes panels with ChatGPT, saving 2-ish hours/week |
| β’ Can explain privacy limits (e.g., no PII in public models) | |
| Adoptive (3) | β’ Automates onboarding docs; runs LLM resume-screen with bias checks, yielding 3Γ faster shortlists |
| β’ Measures time-to-hire gains and refines prompts for under-represented talent pools | |
| Transformative (4) | β’ Revamps recruiting funnel with AI to shorten time-to-hire by 30% |
| β’ Trains HRBPs on safe AI and shapes company policy on ethical hiring AI |
Assessment Questions:
- What AI tools do you use in recruiting or HR operations today?
- How do you ensure AI screening doesn't introduce bias?
- What's your view on AI's role in performance management?
1.5 Marketing
| Level | Indicators |
|---|---|
| Unacceptable (1) | β’ Runs campaigns without AI-driven A/B tests or content variants |
| β’ Ignores AI tools for analytics, personalization, or audience insights | |
| Capable (2) | β’ Uses AI to summarize customer stories |
| β’ Drafts first drafts of social posts and headlines with AI, then edits by hand | |
| Adoptive (3) | β’ Runs a basic AI stack and A/B-tests copy to increase CTR by 30% |
| β’ Audits biased language and optimizes prompt libraries | |
| Transformative (4) | β’ Builds an AI-driven campaign engine to personalize content at scale |
| β’ Leads quarterly AI trainings, sets tooling roadmap, and speaks at industry events on AI-powered growth |
Assessment Questions:
- How do you use AI in content creation or campaign optimization?
- Describe an A/B test where AI-generated content outperformed human-written.
- How do you balance brand voice consistency with AI-generated content?
Section 2: Additional Functions
2.1 Finance
| Level | Indicators |
|---|---|
| Unacceptable (1) | β’ Manual data entry for all reconciliations |
| β’ Exports to Excel for every analysis; no automation | |
| β’ Dismisses AI as "not accurate enough for finance" | |
| Capable (2) | β’ Uses AI to draft variance explanations and summarize reports |
| β’ Can prompt for basic financial analysis (trend identification, anomaly flagging) | |
| Adoptive (3) | β’ Builds automated reconciliation workflows with exception flagging |
| β’ Uses AI for cash flow forecasting with human validation checkpoints | |
| β’ Measures time savings and accuracy improvements | |
| Transformative (4) | β’ Deploys AI-powered FP&A that surfaces insights executives act on |
| β’ Builds predictive models for revenue/churn that inform strategy | |
| β’ Shapes org-wide AI governance for financial data |
Assessment Questions:
- Where do you see AI fitting (or not fitting) in financial workflows?
- How would you validate an AI-generated forecast before presenting to leadership?
- What controls would you require for AI touching financial data?
2.2 Operations
| Level | Indicators |
|---|---|
| Unacceptable (1) | β’ All processes manual and undocumented |
| β’ No interest in automation; "we've always done it this way" | |
| β’ Reactive to issues rather than predictive | |
| Capable (2) | β’ Uses AI to document processes and create SOPs |
| β’ Basic workflow automation (Zapier, Power Automate) for simple tasks | |
| Adoptive (3) | β’ Deploys AI for demand forecasting, inventory optimization |
| β’ Builds exception-based workflows (AI handles routine, humans handle exceptions) | |
| β’ Tracks operational metrics and iterates on AI accuracy | |
| Transformative (4) | β’ Runs AI-optimized operations with real-time adjustment |
| β’ Builds predictive maintenance or demand systems that drive competitive advantage | |
| β’ Leads cross-functional AI ops initiatives |
Assessment Questions:
- What's the most repetitive process in your area? How might AI help?
- How do you handle exceptions when automation fails?
- Describe an operational improvement driven by data/AI insights.
2.3 Sales
| Level | Indicators |
|---|---|
| Unacceptable (1) | β’ No CRM discipline; manual tracking |
| β’ Cold outreach is copy-paste templates | |
| β’ Dismisses AI as "can't replace relationships" | |
| Capable (2) | β’ Uses AI to draft personalized outreach and follow-ups |
| β’ Summarizes call notes with AI; logs to CRM | |
| β’ Understands basic lead scoring concepts | |
| Adoptive (3) | β’ Deploys AI lead scoring that prioritizes pipeline effectively |
| β’ Uses conversation intelligence (Gong, Chorus) and acts on insights | |
| β’ Measures conversion lift from AI-assisted outreach | |
| Transformative (4) | β’ Builds AI-powered sales playbooks that measurably improve win rates |
| β’ Deploys predictive deal scoring that sales leadership relies on | |
| β’ Shapes go-to-market strategy with AI-derived customer insights |
Assessment Questions:
- How do you use AI in prospecting or deal management today?
- Describe a deal where AI insights changed your approach.
- How would you measure ROI on a sales AI tool?
2.4 Legal / Compliance
| Level | Indicators |
|---|---|
| Unacceptable (1) | β’ Reviews every contract manually, line-by-line |
| β’ No document automation or templates | |
| β’ Refuses AI due to confidentiality concerns (without evaluating solutions) | |
| Capable (2) | β’ Uses AI to summarize contracts and flag key terms |
| β’ Understands confidentiality requirements for AI tools | |
| β’ Drafts first-pass documents with AI, then reviews | |
| Adoptive (3) | β’ Deploys contract analysis AI with defined approval workflows |
| β’ Tracks time savings and reviews accuracy rates | |
| β’ Builds prompt libraries for common legal tasks | |
| Transformative (4) | β’ Shapes org-wide AI policy and acceptable use guidelines |
| β’ Deploys AI contract review that handles 80% of routine agreements | |
| β’ Advises on AI risk, IP, and regulatory compliance |
Assessment Questions:
- What's your view on AI for contract review? What safeguards would you require?
- How do you evaluate confidentiality risks of AI tools?
- What AI governance policies should the organization have?
Section 3: Scoring Methodology
3.1 Individual Assessment
Rate each individual 1-4 based on the rubric indicators. Use evidence from:
- Self-assessment questionnaire
- Manager input
- Work samples / tool usage data
- Interview responses
Scoring:
| Score | Level | Description |
|---|---|---|
| 1 | Unacceptable | Immediate training need; potential blocker |
| 2 | Capable | Foundation in place; ready for advancement |
| 3 | Adoptive | Strong contributor; potential champion |
| 4 | Transformative | Leader; can drive org-wide change |
3.2 Team/Function Assessment
Aggregate individual scores to assess team readiness:
Team AI Fluency Score = Ξ£(Individual Scores) / N
Interpretation:
β’ 1.0 - 1.5: Critical gap - transformation at risk
β’ 1.5 - 2.0: Significant gap - heavy training investment needed
β’ 2.0 - 2.5: Moderate gap - targeted upskilling required
β’ 2.5 - 3.0: Adequate - ready for transformation with support
β’ 3.0 - 3.5: Strong - can lead initiatives
β’ 3.5 - 4.0: Exceptional - competitive advantage
3.3 Distribution Analysis
Beyond averages, examine the distribution:
| Distribution Pattern | Implication | Action |
|---|---|---|
| Bimodal (1s and 3s) | Adopters vs. resisters | Address resistance; leverage champions |
| Left-skewed (mostly 1-2) | Org-wide capability gap | Broad training program |
| Right-skewed (mostly 3-4) | Ready for acceleration | Deploy advanced initiatives |
| Uniform | Mixed maturity | Tiered training approach |
Section 4: Assessment Process
4.1 Audit Phase Integration
This assessment is conducted during the Audit phase of Engagement Model: Concurrent Discovery, Coordinated Execution:
Week 1-2 of Audit:
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β 1. Deploy self-assessment survey (all target roles) β
β 2. Conduct manager interviews (calibration) β
β 3. Review tool usage data where available β
β 4. Score individuals and aggregate to teams β
β 5. Identify champions, gaps, and blockers β
β 6. Input to Strategic Readiness Assessment β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
4.2 Self-Assessment Survey
Deploy this survey to all employees in target functions:
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β AI FLUENCY SELF-ASSESSMENT β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β Name: _____________________ Role: _________________________ β
β Department: _______________ Manager: ______________________ β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β 1. Which AI tools do you use regularly? (check all that apply) β
β β‘ ChatGPT / Claude / other LLM β
β β‘ GitHub Copilot / Cursor / other coding assistant β
β β‘ AI features in existing tools (Notion AI, etc.) β
β β‘ Specialized AI tools for my function β
β β‘ None β
β β
β 2. How often do you use AI tools in your work? β
β β‘ Daily β‘ Weekly β‘ Monthly β‘ Rarely β‘ Never β
β β
β 3. Rate your confidence with AI (1=low, 5=high): β
β Understanding how LLMs work: 1 2 3 4 5 β
β Prompting effectively: 1 2 3 4 5 β
β Validating AI outputs: 1 2 3 4 5 β
β Building AI-powered workflows: 1 2 3 4 5 β
β Teaching others to use AI: 1 2 3 4 5 β
β β
β 4. Describe a recent example where AI helped your work: β
β ________________________________________________________ β
β ________________________________________________________ β
β β
β 5. What prevents you from using AI more? (check all that apply) β
β β‘ Don't know what tools to use β
β β‘ Don't have time to learn β
β β‘ Concerned about accuracy/quality β
β β‘ Security/compliance concerns β
β β‘ Don't see the value β
β β‘ Management doesn't support it β
β β‘ Nothing - I use it extensively β
β β
β 6. What AI training would be most valuable to you? β
β ________________________________________________________ β
β β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
4.3 Manager Calibration Interview
Use these questions to calibrate self-assessments:
For each direct report: Where would you place them on the Unacceptable β Transformative scale? What evidence supports this?
Team overall: What's your team's biggest AI capability gap?
Champions: Who on your team could lead AI adoption efforts?
Blockers: Is anyone actively resistant to AI? What's driving that?
Training needs: What would most accelerate your team's AI fluency?
4.4 Scoring Workshop
After collecting data, conduct a calibration session:
Participants: Function leads, HR, Transformation PMO
Process:
- Review aggregated self-assessment data
- Incorporate manager calibration
- Assign final scores (1-4) for each individual
- Calculate team averages
- Identify:
- Champions (score 3-4): Potential leaders for AI initiatives
- Ready (score 2-3): Good candidates for upskilling
- At-risk (score 1-2): Need intervention or may not fit future state
- Blockers (score 1 + resistance): Require direct conversation
Section 5: Output and Integration
5.1 AI Fluency Heatmap
Visualize results across the organization:
UNACCEPTABLE CAPABLE ADOPTIVE TRANSFORMATIVE
1 2 3 4
βββββββββββββββββββββββββββββββββββββββββββββββββ
ENGINEERING ββββββββββββββββββββββββββββββββββββββββββββββ
β² 2.8 avg
PRODUCT ββββββββββββββββββββββββββββββββββββββββββββββ
β² 3.1 avg
SUPPORT ββββββββββββββββββββββββββββββββββββββββββββββ
β² 1.9 avg
PEOPLE / HR ββββββββββββββββββββββββββββββββββββββββββββββ
β² 2.3 avg
MARKETING ββββββββββββββββββββββββββββββββββββββββββββββ
β² 2.7 avg
FINANCE ββββββββββββββββββββββββββββββββββββββββββββββ
β² 2.1 avg
OPERATIONS ββββββββββββββββββββββββββββββββββββββββββββββ
β² 2.4 avg
SALES ββββββββββββββββββββββββββββββββββββββββββββββ
β² 2.6 avg
LEGAL ββββββββββββββββββββββββββββββββββββββββββββββ
β² 1.8 avg
βββββββββββββββββββββββββββββββββββββββββββββββββ
ORG AVERAGE β² 2.4
5.2 Gap Analysis Report
Summarize findings for leadership:
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β AI FLUENCY GAP ANALYSIS β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β β
β EXECUTIVE SUMMARY β
β Organization AI Fluency Score: 2.4 / 4.0 (Capable) β
β Target for Transformation: 2.8+ (Adoptive baseline) β
β Gap: 0.4 points across organization β
β β
β CRITICAL GAPS (Score < 2.0) β
β β’ Support: 1.9 - Manual processes, no AI integration β
β β’ Legal: 1.8 - Resistance due to confidentiality concerns β
β β
β STRENGTHS (Score β₯ 3.0) β
β β’ Product: 3.1 - Strong AI-native mindset β
β β
β CHAMPIONS IDENTIFIED β
β β’ Engineering: 2 Transformative (4), 5 Adoptive (3) β
β β’ Product: 3 Transformative (4), 4 Adoptive (3) β
β β’ Marketing: 1 Transformative (4), 3 Adoptive (3) β
β β
β BLOCKERS IDENTIFIED β
β β’ Support: 3 individuals (Score 1, active resistance) β
β β’ Legal: 2 individuals (Score 1, compliance concerns) β
β β
β RECOMMENDED ACTIONS β
β 1. Deploy broad AI fundamentals training (all functions) β
β 2. Function-specific deep dives (Support, Legal priority) β
β 3. Champion network formation (leverage 3s and 4s) β
β 4. Direct conversations with identified blockers β
β 5. Re-assess in 90 days β
β β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
5.3 Integration with Other Frameworks
Strategic Readiness Assessment (SRA):
- AI Fluency Score feeds into Human Capital dimension
- Gap analysis informs training investment in business case
Crystallization Toolkit:
- Champions (3-4) are prime candidates for expansion redeployment
- Adoptive (3) individuals can absorb work from consolidated roles
- Training needs inform crystallization timelines
Metrics Framework:
- AI Fluency Score is a leading indicator for transformation success
- Track score progression as lagging indicator of training effectiveness
Redeployment Planning:
- Match fluency levels to expansion role requirements
- Identify reskilling paths for high-potential individuals
Section 6: Progress Tracking
6.1 Reassessment Cadence
| Milestone | Assessment Type | Purpose |
|---|---|---|
| Audit (Day 0) | Full assessment | Baseline |
| Commit + 30 days | Champions only | Validate readiness |
| Execute + 90 days | Full assessment | Progress check |
| Deploy | Full assessment | Verify capability for sustainment |
| Ongoing (quarterly) | Pulse check | Continuous improvement |
6.2 Progress Dashboard
Track AI fluency over time:
| Function | Baseline | +90 Days | +180 Days | Target | Status |
|---|---|---|---|---|---|
| Engineering | 2.8 | 3.1 | 3.3 | 3.5 | π’ On track |
| Product | 3.1 | 3.2 | 3.4 | 3.5 | π’ On track |
| Support | 1.9 | 2.3 | 2.6 | 3.0 | π‘ Progressing |
| People/HR | 2.3 | 2.5 | 2.7 | 3.0 | π‘ Progressing |
| Marketing | 2.7 | 2.9 | 3.1 | 3.5 | π’ On track |
| Finance | 2.1 | 2.4 | 2.6 | 3.0 | π‘ Progressing |
| Operations | 2.4 | 2.7 | 2.9 | 3.0 | π’ On track |
| Sales | 2.6 | 2.8 | 3.0 | 3.0 | π’ On track |
| Legal | 1.8 | 2.1 | 2.4 | 2.5 | π‘ Progressing |
| ORG AVG | 2.4 | 2.7 | 2.9 | 3.0 | π’ |
Appendix A: Quick Reference
When to Use This Tool
| Situation | Use This Tool To... |
|---|---|
| Starting an engagement | Baseline organizational AI fluency |
| Planning training | Identify gaps and priorities by function |
| Staffing expansion initiatives | Find champions and qualified candidates |
| Crystallization planning | Assess redeployment readiness |
| Quarterly reviews | Track progress and adjust interventions |
Key Deliverables
- AI Fluency Heatmap - Visual of org-wide capability
- Gap Analysis Report - Executive summary with recommendations
- Champion Registry - List of high-fluency individuals for leverage
- Training Plan - Prioritized upskilling roadmap
- Progress Dashboard - Tracking over time
Appendix B: Customization Guide
Adding Functions
To add a new function to the rubric:
- Define the four levels (Unacceptable β Transformative)
- Identify 3-4 observable indicators per level
- Create 3 assessment questions
- Add to self-assessment survey
- Include in scoring workshop
Adapting for Industry
Adjust indicators based on industry context:
- Healthcare: Add HIPAA compliance considerations
- Financial Services: Emphasize regulatory/audit trail requirements
- Manufacturing: Include operational technology / IIoT context
- Professional Services: Focus on client-facing AI applications
Weighting Functions
If some functions are more critical to transformation:
Weighted Org Score = Ξ£(Function Score Γ Function Weight) / Ξ£(Weights)
Example weights:
- Engineering: 1.5 (critical path)
- Product: 1.5 (critical path)
- Support: 1.0 (standard)
- Others: 1.0 (standard)