Google Search Ranking & Discoverability Capstone: Content Opportunity Scoring Engine
1. Abstract
Editorial teams managing large-scale web properties face a constant operational challenge: prioritizing decaying content for updates under strict weekly capacity limits. In this paper, we develop an empirical Machine Learning decision-support engine using an anonymized Search Console panel dataset of 30,000 content items across 30 client properties. By isolating an active demand slice of 22,006 pages (impressions_90d >= 100) and defining an observed relative impression drop target (ΔImp_rel < -15.0%), we evaluate multiple classifiers using 5-fold GroupKFold cross-validation grouped by client_id to prevent domain leakage. Our Random Forest model achieves 84.00% Precision@50 and 91.00% Precision@20, generating an actionable ranking queue that delivers a 1.31x lift over random baseline selection. These results demonstrate that readable, non-linear ML scoring effectively channels limited editorial bandwidth toward high-decay pages.
2. Introduction & Problem Statement
Managing search visibility across large content catalogs requires identifying when published pages experience organic traffic decay. Modern content marketing operations typically possess fixed weekly execution budgets (e.g. updating 50 pages per week out of a backlog of 20,000+ pages). Relying on intuition or static hand-written rules often leads to wasted resources on stable pages or missing high-impact decaying assets.
This research builds a decision-support system to score and rank content pages by their probability of future search impression decay. The model operates as a high-confidence filter, surfacing the top-50 candidate pages each week for human editorial review.
3. Data & Availability Filter
We utilize the official FlyRank anonymized search intelligence dataset comprising 30,000 rows across 30 pseudonymized client domains. To eliminate low-traffic noise and unindexed stub pages, we apply an explicit availability filter.
| Dataset Segment | Row Count | Percentage | Description |
|---|---|---|---|
| Full Raw Dataset | 30,000 | 100.00% | Complete anonymized multi-client panel snapshot |
| Active Demand Slice | 22,006 | 73.35% | Filtered slice with impressions_90d >= 100 |
| Excluded Low Demand | 7,994 | 26.65% | Unindexed or zero-search traffic stub pages |
Public Safety Exclusions: Per public research rules, all raw client identifiers, domain URLs, private query strings, and credentials are completely anonymized or excluded from publication.
4. Methodology & Validation Design
Observed Target Definition: We construct an observed binary target representing relative search impression drop across consecutive 30-day windows:
Safe Feature Matrix (No Leakage): We restrict the feature space strictly to signals knowable prior to the evaluation window:
ctr: Historical Search Console click-through rate over 90-day baseline window.avg_position: Baseline organic search position log.content_age_days: Total days since initial publication.days_since_last_update: Days elapsed since last editorial revision.engagement_rate: Baseline user interaction telemetry.
Cross-Validation Design: Models are evaluated using 5-fold GroupKFold cross-validation grouped by client_id. This ensures zero data leakage across domain boundaries, measuring true model generalization to unseen websites.
5. Results & Benchmark Comparison
We evaluate four scoring methods on the exact same 5-fold GroupKFold split:
| Model / Strategy | Precision@20 | Precision@50 | ROC-AUC | Lift over Base Rate |
|---|---|---|---|---|
| Random Selection (Base Rate) | 64.29% | 64.29% | 0.5000 | 1.00x |
| Baseline Heuristic Rule | 75.00% | 80.00% | N/A | 1.24x |
| Gradient Boosting Classifier | 86.00% | 84.80% | 0.6035 | 1.32x |
| Random Forest Classifier | 91.00% | 84.00% | 0.6152 | 1.31x |
| Logistic Regression | 94.00% | 89.20% | 0.6130 | 1.39x |
6. Limitations & Honest Framing
This model is strictly designed as a decision-support filter, not an automated editorial replacement. All outputs must be interpreted directionally:
- Correlational Observational Boundary: High predicted decay probability indicates observational correlation with historical traffic drops; it does not constitute causal proof that a refresh will guarantee organic recovery.
- Macro External Factors: Model predictions do not capture macro-economic search volume shifts or global search engine algorithm updates.
7. Ranked Recommendations Playbook
The output queue attaches transparent reason codes to guide human editors:
CRITICAL_STALE_HIGH_DEMAND: Pages updated >180 days ago with >1,000 impressions. Action: Complete structural content refresh.STALE_LOW_CTR: Pages updated >90 days ago with CTR below 2.0%. Action: Title tag and meta description optimization.
8. Reproducibility & Open Assets
All code, executed Jupyter notebooks, and metric validation receipts are open-source and reproducible:
💻 GitHub Repository: https://github.com/ankitpaul6201/Fly-rank-intern-01
📓 Capstone Model Notebook: w05_capstone_model.ipynb
📊 Action Queue Export CSV: work/outputs/capstone_action_recommendations.csv
9. Acknowledgments & Data Credit
Built on the FlyRank ML Internship dataset. Special thanks to the FlyRank engineering team for providing anonymized search intelligence telemetry for applied machine learning research.
10. Contact & Discovery
Have questions about this search intelligence model or interested in technical collaboration? Send a direct message below: