zhang2025-replicationView the repo on GitHub
Pre-registered replication

Does news sentiment earn a Sharpe ratio near 5? Rebuilt from the paper, it earns nothing.

Zhang (2025) reports cost-adjusted out-of-sample Sharpe ratios of 5.87 (EUR/USD), 4.65 (USD/JPY) and 4.65 (10-year Treasury futures) from FinBERT sentiment of GDELT headlines and an XGBoost classifier. This replication followed the paper's event filter, assets, features, models, validation and costs, and registered its design and verdict rules before any result existed.

Saved results, read-only. Every number and figure comes from the repo's results/ folder and paper/ figures. Nothing is run in your browser. This page describes the replication's findings about a published method; it makes no claim about why the original numbers arose.

18 of 18asset, timing and model combinations fail. Sharpe ratios range from -1.00 to +0.02.
-0.43EUR/USD XGBoost Sharpe under the paper's timing, against 5.87 reported
47-52%win rates across every combination, against 66-73% reported
0 of 9XGBoost rows beat the 95th percentile of random-sign trading at the same costs

The paper's claim, and what came back

XGBoost, pooled out of sample from mid-2016 to September 2024, net of costs, using the timing the paper describes (variant P).

Paper figures are as reported (Sharpe and win rate, Table 1 of the paper). Replication figures are from results/F1_results.csv. The one-sided p-value is a 21-day block bootstrap.

All 18 combinations

Three assets, three timing variants and two models. P feeds the model the news dated the same day, as the paper describes. S uses only news published before the close (strict). L feeds the model the next day's news, a deliberate look-ahead. Filter, sort, and compare each Sharpe with the paper's and with random trading.

Asset
Timing
Model
Replication, XGBoostReplication, logisticPaper, reportedTick: 95th percentile of 20 random-sign strategies

Select a dot for detail.

Sharpe is annualised on the pooled out-of-sample daily returns. "Random-sign p95" is the 95th percentile of 20 random-sign strategies on the same days at identical costs. Where the twins are negative (ZN), frequent position changes at 0.05% per round trip are costly.

Growth of $1

The replication's XGBoost strategy under the paper's timing, net of costs, on a log scale. The dashed line is constant growth at the paper's reported CAGR over the same dates, drawn for scale only; it is not the original's equity curve.

Asset

The news-timing hypothesis was rejected, and one candidate explanation

The pre-registered hypothesis was that the reported performance came from timing: the described method would fail while a one-day look-ahead would reproduce it. The data rejected the second half. Variant L, which sees a full day of future news, also earns nothing.

Sharpe ratios by explanation for EUR/USD, USD/JPY and 10-year Treasury futures. The reported values are 5.87, 4.65 and 4.65. The three news timing variants sit near zero within the random-sign band. A price-feature shift on all days gives 17.8, 16.0 and 15.2, and on some days gives 5.7, 5.4 and 3.6.
Sharpe ratio by explanation (replication paper, Figure 1). PDF
Growth of one dollar on a log scale for the replication's XGBoost strategy, flat to slightly down for all three assets, against dashed lines of constant growth at the reported rates.
Growth of $1 against reported growth (replication paper, Figure 2). PDF

Post hoc candidate explanation: a one-row shift in a price feature

Not pre-registered, and not a claim about what the original code did. The original code was not available for comparison, so nothing here shows how its results arose. This is an illustration of magnitude: if the lagged-return feature were shifted by a single row, so that it equalled the next day's return (the target), the same pipeline would learn it directly.

Re-running variant P with only that change gives Sharpe ratios of (EUR/USD), (USD/JPY) and (ZN), with win rates of 91 to 98%, as saved in the daily-returns file. Applying the shift to only a share of days gives numbers of the reported order (5.7, 5.4 and 3.6, per the replication paper's Figure 1). It shows how a one-line error of that kind produces figures of this size, whereas no treatment of the news timing does. Other possible causes could only be assessed with the original code.

How the replication was set up

1. Events and headlinesGDELT 1.0 daily exports, 2015 to October 2024. Event codes 100-199, top 100 per day. Headlines come from article URL slugs; on the 389 of 1,000 sampled links that still had a live title, slug and title FinBERT scores correlate 0.66.
2. Features and modelsFinBERT polarity, daily aggregates, lags, moving averages and the lagged return. Logistic regression and XGBoost under a five-fold expanding-window split. Long if the predicted probability exceeds 0.5, otherwise short, with the paper's costs.
3. Pre-registered verdictsReplicated if variant P has Sharpe of at least 2 on two of three assets. Not replicated, explained by leakage, if P is below 1 everywhere and L at least 2 on two assets. Not replicated, unexplained, if P is below 1 everywhere and L below 2. Outcome: the third.

The full design, committed before any sentiment or model result existed, is in preregistration/PREREGISTRATION.md. An independent replication, Andrews (2026, SSRN 6426038), also found no predictive power, on EUR/USD and other assets. This one adds USD/JPY and 10-year Treasury futures with the paper's event filter, and a direct test of news timing.

Limits, stated plainly

  • A rebuild from the paper's text. The paper's code repository was not available when this was written, so the pipeline was reconstructed. Choices the paper leaves open (hyperparameter grid, inner folds, token limit) are the replication's and are listed in the repo.
  • Headlines are from URL slugs, not scraped pages. Link rot makes scraping a decade of articles impractical. The validation above suggests the daily index is a close proxy, and Andrews (2026), who scraped headlines, also finds no signal.
  • The sample ends in September 2024, not April 2025, because later data is reserved as a hold-out in the author's wider research programme.
  • The one-row shift is an illustration designed after the main results and outside the pre-registered verdict.
  • Not investment advice. This is a methods replication of a published claim.