14.5 · Phân tích dữ liệu với AI

Mở đầu: Câu hỏi đã giữ bạn không dùng AI cho data

"Làm sao tôi biết AI phân tích đúng?"

Đây là câu hỏi đã làm bao nhiêu người dừng lại ngay trước khi upload. Bạn có khảo sát cộng đồng, có donor data, có program outcomes — và bạn biết analysis có thể unlock insights. Nhưng bạn không dám trust AI result vì... bạn không có cách verify.

Thực ra, có một cách — và nó đơn giản đến bất ngờ: test AI against data bạn đã biết câu trả lời.

Đây là essence của Delegation-Diligence loop khi apply vào analytical tasks. Bạn không blind-trust. Bạn không blind-reject. Bạn validate một cách có phương pháp, và build confidence step-by-step.

Trong bài này, bạn sẽ theo dõi Rio, program director tại Valley Veterans Services — một nonprofit phục vụ veterans với housing và job training. Mỗi quý Rio spend nhiều giờ analyze program data. Anh muốn biết AI có thể assist không — hay nên giữ analysis fully manual.

Delegation-Diligence applied to analysis

Nhắc lại từ Bài 14.1:

  ┌──────────────────────────────┐
  │  DELEGATION                  │
  │                              │
  │  Should AI do this task?     │
  │  Which parts?                │
  │  Which tool?                 │
  └──────────────────────────────┘
              ▲
              │ iterate
              ▼
  ┌──────────────────────────────┐
  │  DILIGENCE                   │
  │                              │
  │  Did AI do it right?         │
  │  Can I verify?               │
  │  Do I own final result?      │
  └──────────────────────────────┘

Với analysis, trọng tâm chuyển từ "tôi prompt tốt không" sang "AI output có đúng không". Tức là: Discernment quan trọng hơn Description cho analytical tasks, nhiều khi.

Phương pháp "Known-Answer Testing"

Đây là cách Rio validate AI's analytical capabilities cho công việc cụ thể của anh:

┌────────────────────────────────────────────────┐
│                                                │
│     KNOWN-ANSWER TESTING METHOD                │
│                                                │
│   1. Chọn analytical task                      │
│      thường làm                                │
│              │                                 │
│              ▼                                 │
│   2. Tìm past data đã phân tích trước         │
│      (biết câu trả lời đúng)                   │
│              │                                 │
│              ▼                                 │
│   3. Work với AI reproduce analysis            │
│      using Description-Discernment             │
│              │                                 │
│              ▼                                 │
│   4. Compare AI result với known answer        │
│              │                                 │
│      ┌───────┴───────┐                         │
│      ▼               ▼                         │
│   ✅ MATCH      ❌ MISMATCH                    │
│   → Trust for    → Identify gap                │
│     future       → Refine / or don't delegate  │
│     similar      this task                     │
│                                                │
└────────────────────────────────────────────────┘

Điểm hay: bạn validate công cụ, không chỉ 1 output. Sau khi validated approach, bạn có thể apply với confidence cho new data.

Kịch bản: Rio test AI cho quarterly analysis

Rio mỗi quý phân tích program attendance + employment outcomes cho job training program. Cụ thể:

• Participation rates
• Monthly attendance changes
• Correlation giữa attendance và job placement
• Breakdown theo cohort (enrollment date) và program type

Analysis này thường lấy 6-8 giờ của Rio mỗi quý (data cleaning + formulas + interpretation).

Step 1: Delegation decision

Rio think through:

• Interpretation & decisions: Giữ. Rio muốn continue ra strategic decisions dựa trên data.
• Data cleaning + formulas: Candidates for AI — tedious, error-prone manually.
• Pattern identification: Candidates for AI — potentially faster.
• Contextual nuance: Giữ với Rio — AI không biết lịch sử chương trình, staff changes, ngoại cảnh.

Decision

Test AI cho heavy-lifting phân tích. Keep interpretation.

Platform choice

Claude Team plan (no training on user data). Data có PII participants — cần safe tier. Rio sanitize file trước upload (anonymize participant IDs).

Step 2: Test setup

Rio uploads:

• Last quarter's raw data (sanitized)
• Last quarter's completed analysis (Rio's hand work) — as hidden "answer key"

Wait — Rio không upload answer key. Điểm của test là: AI work từ raw data, Rio check output against known answer offline.

Step 3: First prompt

Tôi share attendance data và employment outcome data từ job training program quý trước. Please: 1. Analyze participation patterns across 3 months 2. Graph correlations giữa attendance levels và employment success 3. Identify whether consistent attendance predicts better job placement outcomes Tôi sẽ compare kết quả của bạn với analysis tôi đã làm trước đó.

AI responds với summary. Rio reviews:

Match:

• Correlation giữa attendance và job placement — số liệu matching Rio's hand analysis

Mismatch:

• AI missed critical insight về combined housing assistance + job placement program — participants trong program kết hợp có outcomes khác hẳn.

Step 4: Iterate — refine description

Rio doesn't reject AI. He refines:

Good on attendance correlation. Nhưng bạn missed pattern quan trọng: participants trong program kết hợp housing assistance + job placement có outcomes khác biệt. Please re-analyze, paying special attention to program type variable trong data.

AI catches it this time. Rio notes: "For future quarters, I need to specifically request program-type consideration."

Step 5: Test harder

Rio probes giới hạn:

Also analyze results based on when participants enrolled (cohort analysis).

AI response:

I don't have enrollment date data in what you uploaded. However, I can help extract it if there's a column I might have missed, or suggest how to add it.

Rio realizes: AI wasn't given that data. He notes: "AI needs explicit enrollment dates để do cohort analysis. Otherwise it'd infer — which I don't want."

Results of testing

Sau test run, Rio has:

• Validated AI can reproduce correlation analysis với right description
• Identified gap: AI needs explicit program-type framing
• Identified gap: AI can't infer enrollment dates — must be in data
• Documented what to include trong prompts cho future quarters

Critically: Rio hasn't blind-trusted AI. He now has tested approach he can confidently apply — with clear notes về data to include và context to add himself.

Next quarter: Rio applies this approach with fresh data. Analysis takes ~1.5 giờ instead of 6-8. Diligence continues: check numbers make sense, take accountability for final report, be transparent về AI role.

Savings: 4-6 giờ per quarter. Confidence level: validated, not blind.

Điều gì nếu bạn không "data-savvy"?

Bạn có thể đọc kịch bản Rio và nghĩ: "Nhưng tôi không có known answer. Tôi never analyzed this data trước đây. Tôi không confident enough to spot errors."

Câu hỏi hợp lý — và AI vẫn có thể giúp, với approach khác.

AI models đặc biệt giỏi với coding tasks — bao gồm data manipulation, Excel formulas, statistical operations. Khi bạn không sure about data analysis:

1. Treat AI as co-analyst, không phải authority

Ask AI explain process, không chỉ give answers:

Tôi có data về [describe]. Tôi muốn hiểu [question]. - What analytical approach phù hợp? - Walk me through methodology step-by-step. - Show the formula/code you'd use. - Explain limitations của approach này.

You learn along the way. AI becomes tutor, not oracle.

2. Ask for explanations của each step

Khi AI gives result:

Explain how you arrived at this number. Show me math. What assumptions did you make?

If AI can't clearly explain reasoning, that's signal to distrust result.

3. Test understanding với simple cases

Start với simple question bạn can verify manually. Build trust từ đó:

From this data, what's the total number of participants?

Verify manually. Match? Good. Try slightly more complex. Build up.

4. Use AI để learn, không chỉ execute

Over time, AI tutors you. Bạn become more "data-savvy". Rio-level validation becomes accessible.

Validation builds confidence — không eliminate responsibility

Sau validation, you still:

• Check numbers make sense against what you know about programs
• Take accountability for final report
• Be transparent về AI role (Creation + Transparency + Deployment Diligence)

Validated ≠ unmonitored. Testing gives you evidence for confidence. Ongoing diligence maintains that confidence.

Bảng so sánh: Analysis alone vs. with AI fluent

Key insight: AI doesn't replace your analytical judgment — it amplifies it. Bottleneck shifts từ manual manipulation sang strategic thinking.

Ví dụ theo ngành — Data analysis tasks nonprofit

Development / Fundraising

Pain: "Analyze donor giving patterns across 5 years — retention, upgrades, churn."

Approach:

• Sanitize donor file (anonymize names, use donor IDs)
• Test AI với past analysis (quý trước bạn đã làm manually)
• Validated approach: cohort retention, LTV ranges, acquisition channel effectiveness
• Apply với full dataset next analysis
• Kết quả: 2 ngày → 4-6 giờ, với deeper insights

Program Evaluation

Pain: "Annual program evaluation — outcomes, costs, demographic breakdowns."

Approach:

• Known answer: last year's evaluation
• Test: can AI reproduce headline findings?
• If validated: scale to this year's data
• Output: evaluation report draft (human polishes interpretation)
• Kết quả: 2 tuần → 3-4 ngày

Health / Social Services Outcomes

Pain: "Patient outcomes data — service utilization, satisfaction, unmet needs."

Approach:

• Strip PHI (per HIPAA/privacy law — consult compliance)
• Aggregate sensitive categorical info
• Test correlations AI identifies against clinical intuition
• Flag for human expert review: any causal-sounding claims
• Kết quả: Faster insights, clinical team involvement preserved

Grants Reporting

Pain: "Quarterly grant reports — service delivery metrics, compliance tracking."

Approach:

• Standardize output format through Projects / custom instructions
• Validate against past approved reports
• AI generates quarterly update, human verifies, submits
• Kết quả: Consistency + time savings

Advocacy / Policy

Pain: "Analyze legislative voting records, donor influence, policy outcomes."

Approach:

• Public data — less privacy concern
• Test AI analysis against published analyses (known answers)
• Use cho tracking systemic patterns
• Kết quả: Richer advocacy briefings in less time

Volunteer Management

Pain: "Volunteer retention analysis — who stays, who leaves, why."

Approach:

• Anonymize volunteer data
• Test correlations (training attendance, role type, tenure → retention)
• Identify risk factors cho churn
• Apply insights to volunteer engagement strategy
• Kết quả: Data-driven retention improvements

Communications / Marketing

Pain: "Analyze email/social engagement to optimize outreach."

Approach:

• Lower stakes data (your own public communications)
• Great starting point cho people new to AI analysis
• Test across known-performing content
• Build voice intuition into data backing
• Kết quả: Evidence-based content strategy

Operations / Executive

Pain: "Dashboard prep cho board meetings — synthesize metrics from multiple sources."

Approach:

• Upload pre-sanitized data từ program, financial, development
• Template prompt reused quarterly
• AI generates dashboard + commentary
• Kết quả: Board-ready in 2 giờ instead of 2 ngày

Prompt templates cho analysis

1. Pattern identification

I'm sharing [dataset type]. Context: [how collected, what represents].

Please analyze and identify:
- Top 3-5 patterns in [dimension of interest]
- Outliers worth investigating
- Potential correlations (NOT claimed causation)

For each pattern:
- Describe pattern specifically
- Cite which data rows/cases support it
- Note confidence level (strong signal vs. suggestive)
- What additional data would strengthen finding

Important: Don't speculate beyond what data shows. Flag 
any inferences as inferences, not facts.

2. Comparison / segmentation

Segment this data by [variable]:
- Group A: [criterion]
- Group B: [criterion]
- Group C: [criterion]

For each group, provide:
- N (sample size)
- Key metrics distribution
- Differences from overall population

Then compare groups:
- Most statistically meaningful differences
- Caveats (small sample sizes, confounds)
- Questions this raises (not answers)

3. Trend analysis

Analyze trends in [metric] over [timeframe].

Look at:
- Monthly/quarterly direction
- Acceleration or deceleration
- Seasonal patterns
- Anomaly periods

Output:
- Trend narrative (plain language, 200 words)
- Data supporting each claim
- 2-3 hypotheses for what driving trend
- What would refute each hypothesis
- Recommended follow-up analysis

4. Correlation vs. causation check

Data shows [observed pattern].

Before I conclude [causation claim], challenge me:
- What alternative explanations could produce this pattern?
- What confounding variables might I be missing?
- What additional data would help establish causation?
- What's weakest link in the current inference chain?

Be skeptical, not confirmatory.

5. Excel formula / code help

I'm trying to [describe goal] in Excel/Sheets.

Current data structure: [describe columns]
Desired output: [describe]

Please:
- Suggest formula approach
- Walk through logic step-by-step
- Show formula with example data
- Explain what each part does
- Flag edge cases that might break it

I want to understand, not just copy-paste.

6. Statistical sanity check

Claim from my analysis: [claim]
Supporting data: [numbers]

Please sanity-check:
- Is the math correct?
- Are sample sizes adequate for confidence?
- Are there obvious confounds I'm missing?
- Is the effect size practically meaningful or trivially significant?
- How would a skeptical reviewer challenge this?

Be direct about weaknesses.

7. Data cleaning

I have messy data. Issues I know about:
- [issue 1: e.g., inconsistent date formats]
- [issue 2: e.g., missing values]

Please:
- Show specific examples of inconsistencies from data
- Suggest standardization approach
- Generate cleaning formula/code
- Flag any ambiguous cases requiring human decision
- Preview results of cleaning (before/after samples)

8. Visualization suggestions

Data: [describe]
Purpose: [presentation to board / funder / community]
Audience sophistication: [basic / moderate / expert]

Recommend 3-5 visualization types that would tell this story:
- Chart type
- What it highlights
- What it obscures
- Accessibility considerations (color blindness, screen readers)
- Tool to create it (Excel, Datawrapper, other)

Rank by effectiveness for this audience.

9. Survey response synthesis

Data: [N] survey responses.
Fields: [describe]

Please synthesize:
- Top 5 themes (with frequency counts)
- Representative quote for each theme (anonymized)
- Outlier responses worth investigating
- Gaps (questions respondents didn't answer, possible reasons)
- Recommendations for next survey (what to ask, what to drop)

Keep person-first, asset-framing language throughout.

10. Outcome / impact analysis

Program: [describe]
Outcomes data: [what tracked]
Baseline: [pre-program or comparison group]

Please analyze:
- Change from baseline by metric
- Statistical significance of change (caveat assumptions)
- Who benefited most / least? Why might that be?
- What change happened but might not be attributable to program?
- What additional evidence would strengthen causal claim?

Frame as evidence-based honest assessment, not marketing narrative.

Anti-patterns — Sai lầm trong data analysis

Accept first AI interpretation as truth

Triệu chứng

Upload data, AI interprets, you use.

Tại sao là sai

First interpretation often misses context, biases, confounds.

Cách đúng

Always challenge: "Alternative explanations? Confounds? What would refute this?"

Upload non-sanitized sensitive data

Triệu chứng

"AI promised no training, so it's OK."

Tại sao là sai

Belt-and-suspenders. Still risks around retention, access, incidents.

Cách đúng

Strip PII even on trusted tools. See Bài 14.4.

Fabricate "AI said X" credibility

Triệu chứng

Claim in report: "AI analysis shows that..."

Tại sao là sai

AI isn't authority. It's tool. Claim obscures who did analysis.

Cách đúng

"Analysis shows..." + transparency footnote about AI assistance.

Assume AI numerical accuracy

Triệu chứng

Paste data, ask "what's the average", use AI answer.

Tại sao là sai

LLMs can make arithmetic errors, especially with larger numbers.

Cách đúng

For critical calculations, use tools that actually compute (Excel, Python, specialized AI với code execution). Or verify manually.

Miss contextual factors only you know

Triệu chứng

AI says "Program X outperforms Program Y" — you publish.

Tại sao là sai

AI doesn't know Program X had 3x the staff support that year, or served a pre-selected population.

Cách đúng

Always overlay your contextual knowledge on AI patterns.

Let AI do full analysis + interpretation

Triệu chứng

AI generates entire program evaluation report. You sign.

Tại sao là sai

Missing your judgment — the part that actually matters for org.

Cách đúng

AI analyzes + drafts. You interpret + decide. Division of labor.

Run same prompt tweaked until getting desired answer

Triệu chứng

AI says Program X didn't work. You rephrase until AI says it did.

Tại sao là sai

Confirmation bias. Dishonest to your community.

Cách đúng

Take negative findings seriously. Ask AI why Program X underperformed.

Ignore AI's expressed uncertainty

Triệu chứng

AI says "data too sparse for confident conclusion". You publish conclusion.

Tại sao là sai

You override AI's appropriate humility.

Cách đúng

When AI flags uncertainty, take seriously. Get more data or caveat finding.

Mẹo nâng cao

Mẹo 1: Adversarial prompting

After AI analysis, ask:

You are now a skeptical reviewer. Critique above analysis. Identify: - 3 weakest conclusions - 2 methodological concerns - 1 ethical concern about framing

Self-adversarial reveals blind spots.

Mẹo 2: Compare 2 AI models

If stakes high:

• Run same prompt trên Claude AND ChatGPT (or Gemini)
• Compare conclusions
• Where they differ, dig in

Different training data / architectures reveal biases of each.

Mẹo 3: Ask AI to estimate confidence

For each conclusion, estimate confidence level: - High confidence (strong, consistent evidence) - Medium (some evidence, confounds possible) - Low (suggestive but tentative) And flag minimum data you'd need to upgrade confidence.

Helps calibrate how much weight to give each finding.

Mẹo 4: Iterate on time-cost vs. nuance

First pass: broad strokes (5 mins). Second pass: refinement on key findings (15 mins). Third pass: sanity checks (10 mins).

Don't spend 2 hours iterating when 30 mins gives 80% value.

Mẹo 5: Archive validated prompts

When you've validated an analytical approach:

• Save the prompt
• Save notes on what AI tends to miss
• Reuse for similar future tasks

Build internal library of "known-good" prompts.

Mẹo 6: Share validation với team

Rio's testing insight applies to team:

• Document what AI handles well / poorly cho your specific data
• Share với colleagues doing similar analysis
• Collective validation > individual trial/error

Áp dụng ngay

Bài tập 1: Messaging analysis (~30 phút) — LOW STAKES

This exercise uses low-stakes data (your own public communications) to practice Description-Discernment loop cho data analysis.

Part I — Gather data:

Collect 10-20 examples of your org's communications — social media posts, email subject lines, newsletter headlines, event announcements. Mix high-performing và lower-performing content.

Part II — Analyze với AI:

Share dataset với AI. Prompt:

Help me identify patterns: - Themes/topics trong higher-performing content - Language, tone, formatting patterns - Gaps giữa what we communicate và what resonates Apply asset-based framing throughout. Cite specific examples supporting each pattern.

Part III — Apply Discernment:

• Do identified patterns match intuition?
• What context is AI missing (audience, goals)?
• Any surprising patterns?

Reflection:

• What are you trying to learn từ this dataset?
• How does high-performing content align với authentic voice + values?
• Are we reaching right audience?

Stretch goal: Ask AI audit how messaging compares to stated mission/values. Find discrepancies. Create messaging guide from analysis.

Bài tập 2: Donor giving patterns (~40 phút) — MEDIUM STAKES

Apply data analysis to fundraising data, building on Bài 14.4 hygiene practices.

Part I — Prepare data:

Use sanitized donor dataset (anonymized từ Bài 14.4 exercise), hoặc prepare new by removing PII. Ensure historical giving across multiple periods.

Part II — Analyze:

Ask AI identify patterns in:

• Donor retention rates over time
• Recurring vs. one-time donation patterns
• Campaign effectiveness comparisons
• Giving trends by amount ranges

Part III — Apply Discernment:

• Do trends match what you know về donor base?
• Is AI focusing only monetary value, missing relationship factors?
• What patterns strengthen donor relationships, not just maximize revenue?

Reflection:

• Costs of implementing efficiency recommendations? (Focusing on majors at expense of smalls → impact on community perception or long-term sustainability?)
• Patterns strengthening relationships beyond giving amounts?

Bài tập 3: Community needs trend analysis (~30 phút) — HIGHER STAKES (STRETCH)

Advanced: combine multiple data sources for predictive analysis.

Part I — Gather sources:

Collect info you use understand community needs:

• Your program data + service requests
• External reports / datasets about community
• News / policy developments affecting constituents

Part II — Analyze emerging patterns:

Ask AI identify:

• Trends in support types requested
• External factors increasing or changing demand
• Gaps between current services and emerging needs

Part III — Rigorous Discernment:

Highest level of critical evaluation:

• AI predictions vs. direct community experience?
• Systemic factors / local context AI missing?
• Values to keep in mind anticipating needs with dignity + respect?

Reflection:

• How approach process responsibly?
• Factors / systemic issues explaining or contextualizing what AI cannot?

Phản xạ bài học

• How did testing AI against known data change your confidence using it for new analysis?
• What gaps/limitations did you identify that shape how you'll delegate analysis in future?
• Which D (Delegation / Description / Discernment / Diligence) felt most stretched in this exercise?

Tóm tắt bài học

Tài liệu tham khảo

• Claude for Data Analysis: https://www.anthropic.com/news
• Claude Code Interpreter features (computational accuracy): https://claude.com
• AI Fluency Lesson 8 — Closer Look at Discernment: Anthropic Academy