Pattern 19: Causal Inference

Intent

Distinguish correlation from causation to understand which interventions, behaviors, and conditions truly cause desired outcomes versus those that merely correlate, enabling evidence-based decision making and avoiding resource waste on ineffective strategies.

Also Known As

• Causality Analysis
• Cause-and-Effect Analysis
• Treatment Effect Estimation
• Counterfactual Reasoning
• A/B Testing Framework

Problem

Correlation is not causation, but we act like it is.

Sarah's "findings": - Families who volunteer 5+ hours have 95% retention - Families who attend 3+ events have 90% retention - Families with annual commitments have 92% retention

Sarah's decisions: - "Let's require 5 hours volunteering!" (to boost retention) - "Let's mandate 3 events!" (to boost retention) - "Let's push annual commitments!" (to boost retention)

The problem: Confounding variables!

Scenario 1: Volunteer Hours - Correlation: Volunteer → High retention (95%) - True causality: Engagement → Both volunteering AND retention - Confound: Engaged families volunteer AND stay (chicken/egg) - Reality: Forcing unmotivated families to volunteer won't help

Scenario 2: Event Attendance - Correlation: Events → High retention (90%) - True causality: Social connections → Both events AND retention - Confound: Families with friends attend events AND stay - Reality: Dragging isolated families to events won't create bonds

Scenario 3: Annual Commitments - Correlation: Annual → High retention (92%) - True causality: Already committed → Choose annual - Confound: Self-selection (only committed families choose annual) - Reality: Pressuring uncertain families into annual = refunds

Without causal inference: - Confuse correlation with causation - Waste resources on ineffective interventions - Miss actual causal mechanisms - Can't learn what truly works - Make decisions on spurious patterns

With causal inference: - Test interventions rigorously (A/B tests, experiments) - Account for confounding variables - Estimate true treatment effects - Invest in what actually works - Build evidence-based playbook

Context

When this pattern applies:

• Making strategic decisions (high stakes)
• Evaluating intervention effectiveness
• Multiple potential causes (need to isolate effects)
• Historical data available for analysis
• Can run experiments (ideal)

When this pattern may not be needed:

• Obvious causality (payment required → access granted)
• Low stakes decisions (cheap to try)
• No confounding variables present
• Can't collect rigorous data

Forces

Competing concerns:

1. Rigor vs Speed - Rigorous experiments take months - Quick observational analysis takes days - Balance: Observational first, experiments for critical decisions

2. Experimental vs Observational - Experiments = gold standard but costly/slow - Observational = fast but confounds remain - Balance: Use best method available for importance

3. Statistical vs Practical - Statistical significance (p < 0.05) - Practical significance (worth the effort?) - Balance: Require both

4. Internal vs External Validity - Internal: Did it work in THIS test? - External: Will it work in the real world? - Balance: Test in representative conditions

5. Complexity vs Interpretability - Complex methods (instrumental variables) - Simple methods (t-tests) - Balance: Start simple, increase complexity as needed

Solution

Build causal inference framework using multiple methods:

Tier 1: Randomized Controlled Trials (RCT) - Gold standard: Random assignment to treatment/control - Eliminates confounding by design - Costly, slow, but definitive

Tier 2: Quasi-Experimental Methods When RCTs not feasible: - Propensity Score Matching - Regression Discontinuity - Difference-in-Differences - Instrumental Variables

Tier 3: Observational with Controls - Regression with covariates - Stratification by confounders - Sensitivity analysis

Framework for causal questions:

1. State hypothesis: "Does X cause Y?"
2. Identify confounders: What else could explain correlation?
3. Choose method: RCT? Quasi-experimental? Observational?
4. Collect data: Treatment, outcome, confounders
5. Estimate effect: Statistical analysis
6. Validate: Robustness checks, sensitivity analysis
7. Interpret: Practical significance, generalizability

Structure

Causal Inference Tables

-- Store experiments (A/B tests, RCTs)
CREATE TABLE experiments (
  experiment_id INT PRIMARY KEY IDENTITY(1,1),

  experiment_name VARCHAR(200) NOT NULL,
  hypothesis NVARCHAR(1000),

  -- Design
  experiment_type VARCHAR(50),  -- 'rct', 'quasi_experimental', 'observational'
  treatment_description NVARCHAR(1000),
  control_description NVARCHAR(1000),

  -- Randomization
  randomization_method VARCHAR(100),  -- 'simple', 'stratified', 'block'
  randomization_seed INT,

  -- Sample size
  planned_sample_size INT,
  actual_sample_size INT,
  power_calculation NVARCHAR(500),

  -- Timeline
  start_date DATE,
  end_date DATE,

  -- Results
  primary_outcome VARCHAR(100),
  treatment_effect DECIMAL(10,4),
  standard_error DECIMAL(10,4),
  p_value DECIMAL(10,8),
  confidence_interval_lower DECIMAL(10,4),
  confidence_interval_upper DECIMAL(10,4),

  -- Interpretation
  statistically_significant BIT,
  practically_significant BIT,
  conclusion NVARCHAR(MAX),

  status VARCHAR(50) DEFAULT 'planned',  -- 'planned', 'running', 'completed', 'cancelled'
  created_date DATETIME2 DEFAULT GETDATE()
);

-- Track experiment assignments
CREATE TABLE experiment_assignments (
  assignment_id INT PRIMARY KEY IDENTITY(1,1),
  experiment_id INT NOT NULL,
  family_id INT NOT NULL,

  -- Assignment
  treatment_group VARCHAR(50),  -- 'treatment', 'control'
  assignment_date DATETIME2 DEFAULT GETDATE(),

  -- Baseline covariates (pre-treatment)
  baseline_engagement_score DECIMAL(5,2),
  baseline_risk_score DECIMAL(5,2),
  baseline_tenure_days INT,

  -- Outcome measurement
  outcome_measured BIT DEFAULT 0,
  outcome_value DECIMAL(10,2),
  outcome_date DATETIME2,

  -- Compliance
  received_treatment BIT,  -- Did they actually get treatment?

  CONSTRAINT FK_assignment_experiment FOREIGN KEY (experiment_id)
    REFERENCES experiments(experiment_id),
  CONSTRAINT FK_assignment_family FOREIGN KEY (family_id)
    REFERENCES families(family_id)
);

-- Store propensity scores for matching
CREATE TABLE propensity_scores (
  score_id INT PRIMARY KEY IDENTITY(1,1),
  family_id INT NOT NULL,

  -- Analysis context
  treatment_variable VARCHAR(100),  -- What we're studying
  analysis_date DATE,

  -- Propensity score (probability of treatment given covariates)
  propensity_score DECIMAL(8,6),

  -- Matched control (if using matching)
  matched_control_id INT,
  match_quality DECIMAL(5,4),

  CONSTRAINT FK_propensity_family FOREIGN KEY (family_id)
    REFERENCES families(family_id)
);

-- Track causal findings
CREATE TABLE causal_findings (
  finding_id INT PRIMARY KEY IDENTITY(1,1),

  -- Question
  causal_question NVARCHAR(500),  -- "Does mentoring cause retention?"

  -- Method used
  analysis_method VARCHAR(100),  -- 'rct', 'propensity_matching', 'regression', etc.
  experiment_id INT,

  -- Estimate
  estimated_effect DECIMAL(10,4),
  standard_error DECIMAL(10,4),
  confidence_interval_lower DECIMAL(10,4),
  confidence_interval_upper DECIMAL(10,4),
  p_value DECIMAL(10,8),

  -- Interpretation
  direction VARCHAR(20),  -- 'positive', 'negative', 'no_effect'
  magnitude VARCHAR(20),  -- 'large', 'medium', 'small'
  practical_significance BIT,

  -- Evidence quality
  evidence_strength VARCHAR(20),  -- 'strong', 'moderate', 'weak'
  confounding_controlled BIT,

  conclusion NVARCHAR(MAX),
  analysis_date DATE,

  CONSTRAINT FK_finding_experiment FOREIGN KEY (experiment_id)
    REFERENCES experiments(experiment_id)
);

Implementation

Randomized Controlled Trial

class RandomizedControlledTrial {
  constructor(db) {
    this.db = db;
  }

  async setupExperiment(config) {
    // Create experiment
    const expResult = await this.db.query(`
      INSERT INTO experiments (
        experiment_name,
        hypothesis,
        experiment_type,
        treatment_description,
        control_description,
        randomization_method,
        planned_sample_size,
        primary_outcome,
        start_date,
        status
      ) VALUES (?, ?, 'rct', ?, ?, ?, ?, ?, CURRENT_DATE, 'planned')
      RETURNING experiment_id
    `, [
      config.name,
      config.hypothesis,
      config.treatmentDescription,
      config.controlDescription,
      config.randomizationMethod || 'simple',
      config.sampleSize,
      config.primaryOutcome
    ]);

    const experimentId = expResult[0].experiment_id;

    // Select eligible families
    const eligibleFamilies = await this.getEligibleFamilies(config.eligibilityCriteria);

    // Power analysis (do we have enough participants?)
    const powerAnalysis = this.calculatePower(eligibleFamilies.length, config.expectedEffect);
    if (powerAnalysis.power < 0.80) {
      console.warn(`Warning: Statistical power only ${(powerAnalysis.power * 100).toFixed(0)}% with ${eligibleFamilies.length} participants. Need ${powerAnalysis.recommendedN} for 80% power.`);
    }

    // Randomize to treatment/control
    const assignments = this.randomize(eligibleFamilies, config.randomizationMethod);

    // Save assignments
    for (const assignment of assignments) {
      await this.db.query(`
        INSERT INTO experiment_assignments (
          experiment_id,
          family_id,
          treatment_group,
          baseline_engagement_score,
          baseline_risk_score,
          baseline_tenure_days
        ) VALUES (?, ?, ?, ?, ?, ?)
      `, [
        experimentId,
        assignment.family_id,
        assignment.group,
        assignment.baseline_engagement_score,
        assignment.baseline_risk_score,
        assignment.baseline_tenure_days
      ]);
    }

    return {
      experiment_id: experimentId,
      n_treatment: assignments.filter(a => a.group === 'treatment').length,
      n_control: assignments.filter(a => a.group === 'control').length,
      power: powerAnalysis.power
    };
  }

  async getEligibleFamilies(criteria) {
    // Build SQL based on eligibility criteria
    let sql = `
      SELECT 
        f.family_id,
        fem.engagement_score as baseline_engagement_score,
        ra.withdrawal_risk as baseline_risk_score,
        DATEDIFF(NOW(), f.enrollment_date) as baseline_tenure_days
      FROM families f
      JOIN family_engagement_metrics fem ON f.family_id = fem.family_id
      LEFT JOIN risk_assessments ra ON f.family_id = ra.family_id
      WHERE 1=1
    `;

    const params = [];

    if (criteria.minEngagement) {
      sql += ` AND fem.engagement_score >= ?`;
      params.push(criteria.minEngagement);
    }

    if (criteria.maxRisk) {
      sql += ` AND ra.withdrawal_risk <= ?`;
      params.push(criteria.maxRisk);
    }

    if (criteria.minTenureDays) {
      sql += ` AND DATEDIFF(NOW(), f.enrollment_date) >= ?`;
      params.push(criteria.minTenureDays);
    }

    return await this.db.query(sql, params);
  }

  randomize(families, method = 'simple') {
    const assignments = [];

    if (method === 'simple') {
      // Simple random assignment (50/50 split)
      for (const family of families) {
        const group = Math.random() < 0.5 ? 'treatment' : 'control';
        assignments.push({ ...family, group });
      }

    } else if (method === 'stratified') {
      // Stratified by engagement level (ensure balance)
      const lowEngagement = families.filter(f => f.baseline_engagement_score < 50);
      const medEngagement = families.filter(f => f.baseline_engagement_score >= 50 && f.baseline_engagement_score < 75);
      const highEngagement = families.filter(f => f.baseline_engagement_score >= 75);

      for (const stratum of [lowEngagement, medEngagement, highEngagement]) {
        for (const family of stratum) {
          const group = Math.random() < 0.5 ? 'treatment' : 'control';
          assignments.push({ ...family, group });
        }
      }

    } else if (method === 'block') {
      // Block randomization (guarantees exactly 50/50)
      const shuffled = families.sort(() => Math.random() - 0.5);
      const midpoint = Math.floor(shuffled.length / 2);

      shuffled.forEach((family, i) => {
        const group = i < midpoint ? 'treatment' : 'control';
        assignments.push({ ...family, group });
      });
    }

    return assignments;
  }

  calculatePower(n, expectedEffect, alpha = 0.05) {
    // Simplified power calculation
    // Real implementation would use statistical libraries

    const effectSize = expectedEffect / 15;  // Cohen's d (assuming SD ~15)
    const criticalZ = 1.96;  // For alpha=0.05, two-tailed

    // Approximate power calculation
    const ncp = effectSize * Math.sqrt(n / 2);  // Non-centrality parameter
    const power = 1 - this.normalCDF(criticalZ - ncp);  // Very rough approximation

    // Recommend sample size for 80% power
    const recommendedN = Math.ceil((2 * Math.pow(criticalZ + 0.84, 2)) / Math.pow(effectSize, 2));

    return {
      power: Math.max(0, Math.min(1, power)),
      recommendedN: recommendedN
    };
  }

  normalCDF(x) {
    // Rough approximation of normal CDF
    return 0.5 * (1 + this.erf(x / Math.sqrt(2)));
  }

  erf(x) {
    // Approximation of error function
    const a1 = 0.254829592;
    const a2 = -0.284496736;
    const a3 = 1.421413741;
    const a4 = -1.453152027;
    const a5 = 1.061405429;
    const p = 0.3275911;

    const sign = x < 0 ? -1 : 1;
    x = Math.abs(x);

    const t = 1 / (1 + p * x);
    const y = 1 - (((((a5 * t + a4) * t) + a3) * t + a2) * t + a1) * t * Math.exp(-x * x);

    return sign * y;
  }

  async analyzeResults(experimentId) {
    // Get outcome data
    const data = await this.db.query(`
      SELECT 
        ea.treatment_group,
        ea.outcome_value,
        ea.baseline_engagement_score
      FROM experiment_assignments ea
      WHERE ea.experiment_id = ?
        AND ea.outcome_measured = 1
    `, [experimentId]);

    const treatment = data.filter(d => d.treatment_group === 'treatment');
    const control = data.filter(d => d.treatment_group === 'control');

    // Calculate means
    const treatmentMean = treatment.reduce((sum, d) => sum + d.outcome_value, 0) / treatment.length;
    const controlMean = control.reduce((sum, d) => sum + d.outcome_value, 0) / control.length;

    const effect = treatmentMean - controlMean;

    // Calculate standard error
    const treatmentVar = this.variance(treatment.map(d => d.outcome_value));
    const controlVar = this.variance(control.map(d => d.outcome_value));
    const pooledVar = ((treatment.length - 1) * treatmentVar + (control.length - 1) * controlVar) / 
                      (treatment.length + control.length - 2);
    const standardError = Math.sqrt(pooledVar * (1/treatment.length + 1/control.length));

    // T-test
    const tStat = effect / standardError;
    const df = treatment.length + control.length - 2;
    const pValue = this.tTestPValue(tStat, df);

    // Confidence interval (95%)
    const tCritical = 1.96;  // Approximate for large samples
    const ciLower = effect - (tCritical * standardError);
    const ciUpper = effect + (tCritical * standardError);

    // Update experiment with results
    await this.db.query(`
      UPDATE experiments
      SET 
        treatment_effect = ?,
        standard_error = ?,
        p_value = ?,
        confidence_interval_lower = ?,
        confidence_interval_upper = ?,
        statistically_significant = ?,
        status = 'completed'
      WHERE experiment_id = ?
    `, [
      effect,
      standardError,
      pValue,
      ciLower,
      ciUpper,
      pValue < 0.05 ? 1 : 0,
      experimentId
    ]);

    return {
      treatment_mean: treatmentMean,
      control_mean: controlMean,
      treatment_effect: effect,
      standard_error: standardError,
      t_statistic: tStat,
      p_value: pValue,
      confidence_interval: [ciLower, ciUpper],
      statistically_significant: pValue < 0.05,
      sample_size: { treatment: treatment.length, control: control.length }
    };
  }

  variance(values) {
    const mean = values.reduce((sum, v) => sum + v, 0) / values.length;
    return values.reduce((sum, v) => sum + Math.pow(v - mean, 2), 0) / (values.length - 1);
  }

  tTestPValue(tStat, df) {
    // Very rough approximation - real implementation would use statistical library
    const abst = Math.abs(tStat);
    if (abst > 3) return 0.001;
    if (abst > 2.5) return 0.01;
    if (abst > 2) return 0.05;
    if (abst > 1.5) return 0.15;
    return 0.50;
  }
}

module.exports = RandomizedControlledTrial;

Propensity Score Matching (Python)

# propensity_matching.py
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import NearestNeighbors

class PropensityScoreMatcher:
    """
    Propensity Score Matching for causal inference from observational data

    Use when you can't randomize but want to estimate treatment effect
    """

    def __init__(self):
        self.propensity_model = LogisticRegression()
        self.matcher = None

    def estimate_propensity_scores(self, X, treatment):
        """
        Estimate probability of receiving treatment given covariates

        X: DataFrame of covariates (engagement, tenure, etc.)
        treatment: Binary array (1=received treatment, 0=control)
        """
        self.propensity_model.fit(X, treatment)
        propensity_scores = self.propensity_model.predict_proba(X)[:, 1]

        return propensity_scores

    def match_samples(self, propensity_scores, treatment, caliper=0.1):
        """
        Match treated units to control units with similar propensity scores

        caliper: Maximum difference in propensity score for match
        """
        treated_idx = np.where(treatment == 1)[0]
        control_idx = np.where(treatment == 0)[0]

        treated_scores = propensity_scores[treated_idx].reshape(-1, 1)
        control_scores = propensity_scores[control_idx].reshape(-1, 1)

        # Find nearest neighbor for each treated unit
        self.matcher = NearestNeighbors(n_neighbors=1)
        self.matcher.fit(control_scores)

        distances, indices = self.matcher.kneighbors(treated_scores)

        # Create matched pairs
        matches = []
        for i, (dist, idx) in enumerate(zip(distances, indices)):
            if dist[0] <= caliper:
                treated_unit = treated_idx[i]
                control_unit = control_idx[idx[0]]
                matches.append({
                    'treated_id': treated_unit,
                    'control_id': control_unit,
                    'distance': dist[0]
                })

        return pd.DataFrame(matches)

    def estimate_treatment_effect(self, matches, outcomes):
        """
        Calculate Average Treatment Effect on the Treated (ATT)

        matches: DataFrame from match_samples
        outcomes: Array of outcome values
        """
        treated_outcomes = outcomes[matches['treated_id']]
        control_outcomes = outcomes[matches['control_id']]

        att = np.mean(treated_outcomes - control_outcomes)
        se = np.std(treated_outcomes - control_outcomes) / np.sqrt(len(matches))

        return {
            'att': att,
            'standard_error': se,
            'confidence_interval': [att - 1.96*se, att + 1.96*se],
            'n_matches': len(matches)
        }

    def check_balance(self, X, treatment, propensity_scores):
        """
        Check covariate balance before/after matching
        """
        treated = X[treatment == 1]
        control = X[treatment == 0]

        balance_stats = {}
        for col in X.columns:
            treated_mean = treated[col].mean()
            control_mean = control[col].mean()
            pooled_std = np.sqrt((treated[col].var() + control[col].var()) / 2)

            # Standardized mean difference
            smd = (treated_mean - control_mean) / pooled_std

            balance_stats[col] = {
                'treated_mean': treated_mean,
                'control_mean': control_mean,
                'smd': smd
            }

        return pd.DataFrame(balance_stats).T

Usage Example

const rct = new RandomizedControlledTrial(db);

// Setup experiment: Does mentoring cause retention?
const experiment = await rct.setupExperiment({
  name: 'Mentoring Impact on Retention',
  hypothesis: 'Families receiving mentor support have 15pp higher retention',
  treatmentDescription: 'Assigned experienced family mentor, monthly check-ins',
  controlDescription: 'Standard support (no mentor)',
  randomizationMethod: 'stratified',
  sampleSize: 100,
  expectedEffect: 15,
  primaryOutcome: 'retention_rate',
  eligibilityCriteria: {
    minEngagement: 40,
    maxRisk: 80,
    minTenureDays: 30
  }
});

console.log(`
Experiment Setup Complete:
  Experiment ID: ${experiment.experiment_id}
  Treatment Group: ${experiment.n_treatment} families
  Control Group: ${experiment.n_control} families
  Statistical Power: ${(experiment.power * 100).toFixed(0)}%
`);

// ... Wait for experiment to run (3-6 months) ...

// Analyze results
const results = await rct.analyzeResults(experiment.experiment_id);

console.log(`
Experiment Results:
  Treatment Mean: ${results.treatment_mean.toFixed(1)}%
  Control Mean: ${results.control_mean.toFixed(1)}%
  Treatment Effect: ${results.treatment_effect.toFixed(1)} percentage points
  95% CI: [${results.confidence_interval[0].toFixed(1)}, ${results.confidence_interval[1].toFixed(1)}]
  P-value: ${results.p_value.toFixed(4)}
  Statistically Significant: ${results.statistically_significant ? 'YES' : 'NO'}

Interpretation:
  ${results.statistically_significant 
    ? `Mentoring causes a ${results.treatment_effect.toFixed(1)}pp increase in retention (p<0.05). This is a real effect, not due to chance.`
    : `No statistically significant effect detected. Either mentoring doesn't work, or sample size too small.`
  }
`);

// Example output:
// Experiment Results:
//   Treatment Mean: 87.3%
//   Control Mean: 72.5%
//   Treatment Effect: 14.8 percentage points
//   95% CI: [8.2, 21.4]
//   P-value: 0.0023
//   Statistically Significant: YES
//   
// Interpretation:
//   Mentoring causes a 14.8pp increase in retention (p<0.05). This is a real effect, not due to chance.

Variations

By Method Rigor

Tier 1: Randomized Controlled Trial - Gold standard - Random assignment eliminates confounding - Expensive, slow, definitive

Tier 2: Quasi-Experimental - Propensity Score Matching - Regression Discontinuity - Difference-in-Differences - Instrumental Variables

Tier 3: Observational + Controls - Multiple regression - Stratification - Sensitivity analysis

By Application

Product Features: - A/B testing - Multivariate testing - Sequential testing

Interventions: - RCTs - Stepped wedge design - Cluster randomization

Policy Evaluation: - Natural experiments - Regression discontinuity - Difference-in-differences

Consequences

Benefits

1. True causality Know mentoring CAUSES retention (not just correlates).

2. Avoid waste Don't invest in interventions that don't work.

3. Optimize resource allocation Double down on what actually works.

4. Evidence-based Decisions backed by rigorous evidence.

5. Confounding controlled Account for alternative explanations.

6. Effect size quantified Not just "it works" but "14.8pp improvement."

Costs

1. Time investment RCTs take 3-6 months to complete.

2. Complexity Requires statistical expertise.

3. Sample size needs Need 100+ participants for power.

4. Ethical constraints Can't always withhold potentially beneficial treatment.

5. External validity Results may not generalize beyond test population.

6. Implementation overhead Tracking assignments, outcomes, compliance.

Sample Code

Simple A/B test analysis:

async function analyzeABTest(testName) {
  const data = await db.query(`
    SELECT 
      treatment_group,
      COUNT(*) as n,
      AVG(outcome_value) as mean,
      STDDEV(outcome_value) as stddev
    FROM experiment_assignments ea
    JOIN experiments e ON ea.experiment_id = e.experiment_id
    WHERE e.experiment_name = ?
      AND ea.outcome_measured = 1
    GROUP BY treatment_group
  `, [testName]);

  const treatment = data.find(d => d.treatment_group === 'treatment');
  const control = data.find(d => d.treatment_group === 'control');

  const effect = treatment.mean - control.mean;
  const pooledStd = Math.sqrt(
    (Math.pow(treatment.stddev, 2) * (treatment.n - 1) +
     Math.pow(control.stddev, 2) * (control.n - 1)) /
    (treatment.n + control.n - 2)
  );

  const se = pooledStd * Math.sqrt(1/treatment.n + 1/control.n);
  const tStat = effect / se;

  return {
    treatment_mean: treatment.mean,
    control_mean: control.mean,
    effect: effect,
    relative_lift: (effect / control.mean) * 100,
    t_statistic: tStat,
    significant: Math.abs(tStat) > 1.96
  };
}

Known Uses

Homeschool Co-op Intelligence Platform - RCT: Mentoring program (14.8pp retention improvement, p<0.05) - A/B test: Payment reminder timing (5.2pp on-time improvement) - Observational: Annual commitment effect (13pp, but self-selection)

Tech Companies: - Google: 1000s of A/B tests annually - Amazon: Test everything - Netflix: Algorithm improvements

Healthcare: - Clinical trials (FDA requirement) - Treatment effectiveness - Drug development

Education: - What Works Clearinghouse - IES randomized trials - EdTech effectiveness studies

Related Patterns

Requires: - Pattern 1: Universal Event Log - outcome data - Pattern 26: Feedback Loop Implementation - tracking experiments

Enables: - Pattern 15: Intervention Recommendation - recommend proven interventions - Pattern 18: Opportunity Mining - validate opportunities - Pattern 24: Template-Based Communication - A/B test messages

Enhanced by: - Pattern 12: Risk Stratification Models - predict treatment heterogeneity - Pattern 13: Confidence Scoring - confidence in causal claims

References

Academic Foundations

• Pearl, Judea (2009). Causality: Models, Reasoning, and Inference (2nd ed.). Cambridge University Press. ISBN: 978-0521895606 - Foundational causal inference text
• Imbens, Guido W., and Donald B. Rubin (2015). Causal Inference for Statistics, Social, and Biomedical Sciences. Cambridge University Press. ISBN: 978-0521885881
• Angrist, Joshua D., and Jörn-Steffen Pischke (2009). Mostly Harmless Econometrics. Princeton University Press. ISBN: 978-0691120355 - Practical causal inference
• Kohavi, Ron, Diane Tang, and Ya Xu (2020). Trustworthy Online Controlled Experiments. Cambridge University Press. ISBN: 978-1108724265 - A/B testing and experimentation

Time Series Pattern Recognition

• Box, G.E.P., et al. (2015). Time Series Analysis: Forecasting and Control (5th ed.). Wiley. ISBN: 978-1118675021
• Hyndman, R.J., & Athanasopoulos, G. (2021). Forecasting: Principles and Practice (3rd ed.). https://otexts.com/fpp3/ - Free online
• Time Series Classification: Bagnall, A., et al. (2017). "The great time series classification bake off." Data Mining and Knowledge Discovery 31(3): 606-660.

Practical Implementation

• tsfresh: https://tsfresh.readthedocs.io/ - Automatic time series feature extraction
• Stumpy: https://github.com/TDAmeritrade/stumpy - Time series pattern search (matrix profile)
• Prophet: https://facebook.github.io/prophet/ - Detect seasonal patterns and anomalies
• tslearn: https://tslearn.readthedocs.io/ - Time series machine learning (DTW, k-means)
• sktime: https://www.sktime.org/ - Unified time series ML framework

Causal Inference Methods

• DoWhy: https://github.com/py-why/dowhy - Python causal inference library by Microsoft
• CausalImpact: https://google.github.io/CausalImpact/CausalImpact.html - Google's R package
• EconML: https://github.com/py-why/EconML - Heterogeneous treatment effects

Related Trilogy Patterns

• Pattern 10: Engagement Velocity Tracking - Velocity as temporal pattern
• Pattern 12: Risk Stratification Models - Use causal evidence in predictions
• Pattern 20: Natural Experiments - Test causality in natural settings
• Volume 3, Pattern 16: Temporal Validation - Validate temporal data

Tools & Services

• InfluxDB: https://www.influxdata.com/ - Time series database
• TimescaleDB: https://www.timescale.com/ - PostgreSQL for time series
• Grafana: https://grafana.com/ - Time series visualization
• Prometheus: https://prometheus.io/ - Monitoring and time series database