agenthinkmesh/gcc_oilgas_ai

GCC Oil & Gas Upstream AI

Ten scikit-learn models for oil and gas tasks framed around Kuwait and the wider GCC: reservoir production, well failure, oil price scenarios, drilling cost, refinery yield, pipeline corrosion, sovereign wealth fund returns, workforce nationalization, carbon intensity and water cut. One script, gcc_oilgas_ai.py, generates a synthetic dataset for each task (12,000 rows each). It then runs a small AutoML step that compares five candidate algorithms with 3-fold cross-validation and saves the best one per task as a joblib pickle. None of the models has seen real operational data. Treat them as demonstrations and templates, not production tools.

Author: AgenThink, Kuwait City

Models

Test metrics are on a 20% held-out split of the same synthetic data (train_test_split(test_size=0.2, random_state=42)). CV is the mean 3-fold cross-validation score on the 80% training split. All numbers come from automl_results.json.

#	File	Task	Type	Selected algorithm	CV score	Test score	Test MAE
1	model_reservoir_prod.pkl	Reservoir Production Forecast	Regression	Ridge	R² 0.9967	R² 0.997	494.77 bopd
2	model_well_failure.pkl	Well Failure Prediction	Classification	GradientBoosting	Acc 0.9530	Acc 0.9571	–
3	model_oil_price.pkl	Oil Price Scenario Modeling	Regression	Ridge	R² 0.9782	R² 0.9782	2.57 USD/bbl
4	model_drilling_cost.pkl	Drilling Cost Optimization	Regression	GradientBoosting	R² 0.9427	R² 0.9409	0.98 MUSD
5	model_refinery_yield.pkl	Refinery Yield Optimization	Regression	GradientBoosting	R² 1.0 ⚠️	R² 1.0 ⚠️	0.0
6	model_pipeline_corrosion.pkl	Pipeline Corrosion Risk	Classification	GradientBoosting	Acc 0.9545	Acc 0.9583	–
7	model_swf_allocation.pkl	Sovereign Wealth Fund Allocation	Regression	Ridge (per JSON) ⚠️	R² 0.7354	R² 0.7473	0.77 pct-pts
8	model_nationalization.pkl	NOC Workforce Nationalization Score	Regression	Ridge	R² 0.8241	R² 0.8202	4.03 pts
9	model_carbon_intensity.pkl	Carbon Emission Intensity	Regression	GradientBoosting	R² 0.8061	R² 0.8096	4.07 kg CO₂e/bbl
10	model_water_cut.pkl	Oilfield Water Cut Prediction	Regression	Ridge	R² 0.9770	R² 0.9786	0.016 (fraction)

The candidates were RandomForest, GradientBoosting, ExtraTrees, DecisionTree, and either Ridge (for regression) or LogisticRegression (for classification). The CV score of every candidate is stored under all_scores in automl_results.json.

⚠️ Known issues. Read these before using models 5 and 7:

• Refinery Yield (model 5) is degenerate. The synthetic target liquid_yield_pct is clipped to the range 0.65–0.97. The terms added before clipping almost always push it past the upper bound, so the target is effectively the constant 0.97. The saved model's trees are all single nodes, and it predicts 0.97 for any input. The R² of 1.0 and MAE of 0.0 are artifacts of that constant target. They do not show predictive skill. RandomForest, ExtraTrees and DecisionTree scored 0.0 in CV for the same reason.
• The SWF Allocation file (model 7) does not match the documented model. automl_results.json lists Ridge as the selected algorithm, with the 20 features shown below. The uploaded model_swf_allocation.pkl is a GradientBoostingRegressor with 13 input features, no stored feature names, n_estimators=120, max_depth=5 and learning_rate=0.08. The training script cannot produce that configuration: it trains every GradientBoosting candidate with 80 estimators and depth 4. The file most likely comes from a different run or project. The metrics above do not apply to it, and its expected inputs are undocumented. Do not use it until it is regenerated.

Dashboard

The script generates this dashboard. It shows the test score for each model, scatter plots of the synthetic data, the top feature importances, and a heatmap of the algorithm comparison.

Repository files

File	Description
README.md	This model card
gcc_oilgas_ai.py	Full pipeline: synthetic data generation, AutoML selection, evaluation, dashboard, and saving of models and results
automl_results.json	For each model: task, type, selected algorithm, CV and test metrics, CV scores of all candidates, ordered feature list, feature importances, and a free-text gcc_note
gcc_oilgas_ai_dashboard.png	Performance and data dashboard (about 1.7 MB)
model_*.pkl (10 files)	Fitted scikit-learn estimators saved with joblib (see the table above). The pickles record scikit-learn 1.8.0 as the training version

Training data

All training data is synthetic. The script generates it with NumPy (np.random.seed(42)) from hand-written distributions: normal, lognormal, beta, exponential, Poisson, binomial and categorical draws. It does not ship the data as CSV; running the script recreates it. Each task has 12,000 independent rows and 20 input features.

• Regression targets are hand-specified linear combinations of a subset of the features, plus Gaussian noise, clipped to a range. That explains why Ridge wins several tasks and why R² values are high.
• Classification targets come from a hand-weighted risk score. A row is labelled 1 if its score is at or above a quantile: 0.82 for well failure, giving about 18% positives, and 0.78 for pipeline corrosion, about 22%.
• Many features do not appear in the target formula and are pure noise. Their feature importances are near zero.
• Categorical codes (for example field_name 1–8, well_type 1–4, rig_type 1–5) are plain integers. Apart from a code comment ("Burgan, Ratawi, Minagish, etc.") for field_name, they have no documented mapping to real categories.
• The gcc_note strings in automl_results.json are contextual remarks written by the author, for example about Burgan field, KOC, KNPC or KIA. The data does not derive or validate them, and they are not sourced.

Features and targets

Feature order matters. The order below matches features in automl_results.json and model.feature_names_in_, except for the mismatched model 7.

1. Reservoir Production: target production_bopd (100–150,000). Features: field_name, reservoir_depth_m, porosity_pct, permeability_md, reservoir_pressure_psi, temperature_f, water_saturation, oil_gravity_api, well_age_years, active_wells, injection_wells, gas_injection_mmscfd, water_injection_bwpd, artificial_lift, eor_applied, decline_rate_pct, h2s_pct, co2_pct, waxing_tendency, opex_usd_bbl
2. Well Failure: target failure (0/1). Features: well_type, well_age_years, depth_m, production_bopd, water_cut_pct, h2s_ppm, sand_production, artificial_lift_type, casing_integrity, last_workover_days, pressure_anomaly, temperature_anomaly, vibration_idx, flow_rate_drop, scale_deposition, corrosion_rate, paraffin_deposition, downhole_sensor_alerts, maintenance_compliance, iot_monitoring
3. Oil Price: target oil_price_usd (15–140). Features: month, year, opec_production_mbpd, us_shale_mbpd, global_demand_mbpd, inventory_change_mb, usd_index, fed_rate_pct, geopolitical_risk_idx, iran_sanctions, opec_cut_announced, saudi_policy, china_demand_idx, global_gdp_growth, renewables_capacity_gw, shipping_disruption, refinery_outage_mbpd, strategic_reserve_release, prev_month_price, volatility_30d
4. Drilling Cost: target drilling_cost_musd (0.5–60). Features: well_type, target_depth_m, formation_hardness, deviation_angle_deg, h2s_present, high_pressure_zone, rig_type, rig_day_rate_kusd, drill_bit_type, mud_type, casing_strings, completion_type, expected_duration_days, npt_days_expected, contractor_experience, weather_risk, logistics_complexity, epc_contract_type, digital_drilling, prev_well_cost_musd
5. Refinery Yield: target liquid_yield_pct (a fraction, effectively constant at 0.97; see known issues). Features: crude_api, crude_sulfur_pct, crude_blend_ratio, throughput_bpd, cdu_temp_c, fcc_conversion_pct, hydrocracker_severity, reformer_temp_c, catalyst_age_days, hydrogen_availability, fuel_gas_self_consumption, steam_system_efficiency, ambient_temp_c, cooling_water_temp, unit_downtime_hrs_mo, energy_intensity, product_slate_gasoline, product_slate_diesel, maintenance_compliance, advanced_process_control
6. Pipeline Corrosion: target high_corrosion_risk (0/1). Features: pipe_material, pipe_age_years, diameter_inches, operating_pressure_psi, h2s_ppm, co2_pct, water_content_pct, flow_velocity_ms, temperature_c, soil_resistivity, cathodic_protection, coating_condition, external_corrosion_rate, internal_inspection_days, sand_erosion, chloride_ppm, microbiological_activity, stress_corrosion_risk, repair_history_count, smart_pig_days_ago
7. SWF Allocation (as the script documents it; the uploaded file differs): target target_return_pct (0–20). Despite the task name, the model predicts a target return, not an allocation. Features: asset_class_enc (a LabelEncoder over 7 asset classes, sorted alphabetically: Alternatives, Fixed_Income, Global_Equities, Hedge_Funds, Infrastructure, Private_Equity, Real_Estate; the encoder is not saved), oil_price_usd, gcc_budget_surplus, global_risk_idx, us_10yr_yield, usd_index, inflation_global, geopolitical_risk, time_horizon_years, current_allocation_pct, benchmark_deviation, shariah_screen, liquidity_need_12mo, esg_mandate, concentration_limit, currency_hedge_pct, market_timing_signal, gdp_growth_target, fund_size_busd, rebalancing_trigger
8. Nationalization Score: target nationalization_score (10–100). Features: department, role_level, national_headcount, total_headcount, current_national_pct, target_national_pct, training_budget_kusd, scholarship_count, succession_plans, retention_rate, attrition_risk, skill_gap_score, mentorship_programs, specialized_roles, govt_mandate_pressure, salary_competitiveness, work_culture_score, leadership_development, women_in_workforce, expat_dependency_idx
9. Carbon Intensity: target co2e_kg_bbl (5–120). Features: operation_type, production_bopd, gas_flaring_mmscfd, gas_utilization_pct, venting_rate_mmscfd, energy_intensity_gj_bbl, fleet_diesel_consumption, offshore_operations, solar_capacity_mw, carbon_capture_active, methane_leak_rate, crude_type_heaviness, refinery_included, legacy_equipment_pct, maintenance_quality, esg_targets_set, carbon_price_usd, api_gravity, h2s_content, electrification_pct
10. Water Cut: target future_water_cut_pct (a fraction, 0.01–0.98). Features: field_age_years, reservoir_type, current_wc_pct, production_bopd, water_injection_bwpd, injection_to_production_ratio, permeability_md, reservoir_heterogeneity, sweep_efficiency, coning_risk, well_spacing_m, completion_type, viscosity_cp, wettability, aquifer_strength, fracture_network, eor_type, production_rate_drop, months_ahead, workovers_done

For Ridge models, the feature_importance values in the JSON are normalized absolute coefficients. They depend on feature scale, because no scaling is applied, so they are not comparable to tree-based importances.

How to use

The models are plain scikit-learn estimators saved with joblib. Use scikit-learn 1.8.x to avoid version warnings or incompatibilities when unpickling.

# pip install scikit-learn==1.8.* pandas joblib huggingface_hub
import json, joblib, pandas as pd
from huggingface_hub import hf_hub_download

repo = "agenthinkmesh/gcc_oilgas_ai"
results = json.load(open(hf_hub_download(repo, "automl_results.json"), encoding="utf-8"))
model = joblib.load(hf_hub_download(repo, "model_well_failure.pkl"))

features = results["well_failure"]["features"]   # same order as model.feature_names_in_
row = {
    "well_type": 1, "well_age_years": 28.0, "depth_m": 2600.0, "production_bopd": 1500.0,
    "water_cut_pct": 0.45, "h2s_ppm": 20.0, "sand_production": 1, "artificial_lift_type": 2,
    "casing_integrity": 1, "last_workover_days": 900.0, "pressure_anomaly": 1,
    "temperature_anomaly": 0, "vibration_idx": 0.4, "flow_rate_drop": 0.3,
    "scale_deposition": 0.2, "corrosion_rate": 0.3, "paraffin_deposition": 0.1,
    "downhole_sensor_alerts": 3, "maintenance_compliance": 0.7, "iot_monitoring": 1,
}
X = pd.DataFrame([row], columns=features)
print("failure probability:", model.predict_proba(X)[0, 1])

Regression models work the same way: build a one-row DataFrame with that model's features and call model.predict(X). Loading a pickle can execute arbitrary code, so only load files from sources you trust.

Re-running training

pip install numpy pandas matplotlib scikit-learn joblib
# The output directory is hard-coded: edit OUT = "/home/claude/gcc_oilgas_ai" near the top of the script first
python gcc_oilgas_ai.py

The script regenerates the data, all 10 models, automl_results.json, the dashboard and a short README.md. It will overwrite a README in the output directory. Fixed seeds are used, but results can still differ slightly across library versions.

Intended use

• Examples and starting templates for tabular AutoML on oil and gas–style problems.
• Teaching and prototyping, for example to show feature schemas, a model-selection workflow or dashboarding.
• A scaffold to retrain on real, properly governed operational data.

Out of scope: operational decisions such as well interventions, drilling budgets, pipeline integrity management, trading, investment allocation or workforce planning. The models reflect only the formulas in the generator.

Limitations and risks

• Synthetic data only. Every relationship the models learn was written by hand in gcc_oilgas_ai.py. High scores show that the models recover those formulas. They say nothing about real-world validity, and results on real field, market or HR data are unknown.
• Evaluation is in-distribution. Test sets are drawn from the same generator as training. There is no temporal, out-of-distribution or real-data evaluation. Only accuracy is reported for classifiers, with no precision, recall or calibration, even though positive classes are about 18–22%.
• Degenerate and mismatched artifacts. See the known issues for models 5 and 7.
• No affiliation. Organizations named in comments or notes (KOC, KNPC, KIA, OPEC) did not supply data and have not endorsed these models.
• Oil price model (model 3): its inputs include hypothetical scenario flags. It is not a forecasting model, and it must not be used for trading or fiscal decisions.
• Workforce nationalization model (model 8): it scores units using nationality-related variables such as current_national_pct and expat_dependency_idx. Any real use in HR decisions requires legal review, fairness assessment and human oversight.
• Unscaled linear and logistic models: coefficients and "importances" depend on feature scale.

License

MIT

Citation

@misc{agenthink2026gccoilgas,
  author       = {AgenThink},
  title        = {GCC Oil \& Gas Upstream AI},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/agenthinkmesh/gcc_oilgas_ai}}
}