Study snapshot: September 20, 2026. Article prepared on October 22, 2026. All results use data frozen through September 18, 2026. No later prices were added, and no models were retrained for this article.
Adding news to a stock-selection system starts with an appealing idea. Prices and volume describe trading that has already happened. News describes developments as they unfold. Could turning those developments into numbers help a model identify opportunities earlier?
We tested that idea in the trade_agent project. Tiingo supplied the news. JEV scored the text. Our pipeline aggregated those scores into stock-level event factors, which we evaluated in classifiers, ranking models, and individual factor portfolios.
The results were mixed in a revealing way.
Adding the full event feature set did not establish a stable improvement in either model’s primary metric. Individual factors—materiality, reversed mean JEV score, and negative-news share—showed positive portfolio alpha. But after comparison with portfolios based on fixed stock rankings, the incremental value of updating news scores each day remained unconfirmed.
This article explains the experiment, how to interpret its numbers, and why that last comparison matters more than the initial return estimates.
Separating the news feed from the scoring model
Every article in this experiment came through Tiingo. Tiingo was the distribution channel; the publisher labels included Yahoo Finance, CNBC, Bloomberg, and other sources. A single API provider does not imply a single publisher.
JEV handled text scoring. We used TypeSafe’s official API with the version pinned to jev-1.13.0. We did not mix scoring backends or use a moving latest alias. The interface and model versions are described in the TypeSafe documentation.
The project also has adapters for compatible backends. However, the results here concern this pinned official model only; they are not an evaluation of those other implementations.
Each scoring input contained the target ticker, headline, provider-supplied description, source, publication time, and related metadata. We did not assume that the description was the publisher’s complete article. We defined four questions:
| Dimension | Question | Project scale |
|---|---|---|
| Direction | Is the stated business development favorable or adverse? | −1 to 1 |
| Materiality | How consequential is the development for operations, finances, or legal standing? | 0 to 1 |
| Relevance | How directly does the news concern the target company? | 0 to 1 |
| Uncertainty | Is the claim confirmed, conditional, speculative, or disputed? | 0 to 1 |
Swipe to view all columns →
We took the minimum confidence across those four answers as a conservative project-level confidence measure. It measures confidence in the scoring answers, not the probability that the stock will rise.
These questions, mappings, and aggregation rules are our own research design. They are not official JEV financial indicators. We did not ask JEV to predict prices.
From article scores to ten stock-level factors
At five minutes before the New York market open, we aggregated each stock’s visible news over the previous seven calendar days into ten features:
| Factor | Definition |
|---|---|
| News count | All article–stock records in the window |
| Scored count | Successfully scored records in the window |
| Mean JEV score | Mean direction score across scored records |
| Positive-news share | Fraction of scored records with direction above zero |
| Negative-news share | Fraction of scored records with direction below zero |
| Materiality | Mean materiality score |
| Relevance | Mean relevance score |
| Uncertainty | Mean uncertainty score |
| Confidence | Mean of the conservative per-record confidence measure |
| Decayed event impact | Sum of direction × materiality × relevance × confidence, discounted by article age |
Swipe to view all columns →
The decay half-life was one day. News count used the full corpus; score-derived features used only successfully scored sampled records. Unscored records were not assigned neutral scores.
The unit of analysis was an article–stock pair. An article linked to two stocks produced two records. We also did not perform complete semantic deduplication of syndicated stories across publishers. News count therefore does not measure the number of independent underlying events.
Three months of news produced only fourteen evaluable holdout sessions
The experiment used a fixed universe of twenty stocks:
AAPL MSFT NVDA AMZN GOOGL META TSLA JPM BAC GS
XOM CVX UNH JNJ LLY WMT COST HD CAT NEE
The news window ran from June 22 through September 18, 2026. It contained 195 publisher/source labels and 155,689 articles identified by source and article ID. Expanding those articles by ticker produced 207,241 article–stock records.
We used deterministic hash sampling on the article–stock key with a target rate of 20%, completing 41,261 pair-level scores. Selection did not use returns and was reproducible for the same inputs. The realized count need not equal exactly 20% of the corpus.
The nominal design was two months for training and one month for the holdout. News warm-up and the need to observe complete five-session returns reduced the usable sample.
| Purpose | Signal dates in 2026 | Trading sessions | Stock–date rows |
|---|---|---|---|
| Actual training sample | Jun 29–Aug 14 | 34 | 680 |
| Holdout predictions | Aug 24–Sep 18 | 19 | 380 |
| Holdout with complete five-session labels | Aug 24–Sep 11 | 14 | 280 |
| Pending labels; predictions retained only | Sep 14–Sep 18 | 5 | 100 |
Swipe to view all columns →
Returns ran from the adjusted opening price on the signal date to the adjusted opening price five trading sessions later. Training labels had to end strictly before August 24; the latest retained training label ended on August 21. Technical features used information available through the previous trading session.
The 280 holdout rows were not 280 independent observations through time. Stocks on the same date shared market conditions, and adjacent five-session return windows overlapped substantially. There were only fourteen mature holdout sessions.
Experiment one: did news improve the models?
We compared three feature sets while holding samples, targets, hyperparameters, and chronological boundaries constant:
| Variant | Inputs | Features |
|---|---|---|
| Technical baseline | Existing technical-feature allowlist, filtered using training-sample variance | 148 |
| Add news counts | Baseline plus the two count features | 150 |
| Add all event features | Baseline plus all ten event features | 158 |
Swipe to view all columns →
The two model families answered different questions. The histogram gradient-boosting classifier predicted whether a stock’s absolute five-session return would exceed 5%; its primary metric was ROC AUC. XGBRanker used rank:pairwise, grouped by trading date, to learn the ordering of five-session returns relative to QQQ; its primary metric was mean daily Rank IC.
AUC measures how well classification scores distinguish positive from negative observations. Rank IC is the daily rank correlation between scores and future returns, averaged over dates. Neither is a return percentage.
We fixed training at 200 iterations and used seeds 42, 7, and 2026, with 42 designated as primary. We did not select seeds or iteration counts using holdout performance. The primary-seed results were:
| Metric | Technical baseline | Add news counts | Add all events |
|---|---|---|---|
| Classifier AUC, primary | 0.5223 | 0.5958 | 0.5233 |
| Classifier Rank IC, secondary | 0.0799 | 0.1128 | 0.1425 |
| Ranker Rank IC, primary | 0.1560 | 0.0879 | 0.1344 |
| Ranker NDCG@5, secondary | 0.6512 | 0.6603 | 0.6628 |
Swipe to view all columns →
Relative to the technical baseline, adding all event features changed classifier AUC by +0.0011, with a paired 95% interval of [−0.0564, +0.0500]. The ranker’s mean Rank IC changed by −0.0216, with an interval of [−0.0799, +0.0387].
Both primary intervals crossed zero. The higher classifier AUC from counts alone was worth recording, but did not justify selecting that variant for production. The full event variant also improved the classifier’s continuous-return Rank IC, but that secondary metric did not replace the original classification objective.
Intervals came from resampling whole trading dates together in five-session circular blocks, with 2,000 draws. The ranking result was also sensitive to the other fixed seeds: only one of the three produced a Rank IC improvement. All three classifier seeds produced identical predictions, so they did not provide three independent replications.
Experiment two: did individual factors show alpha?
Combining all event features could mix useful, redundant, and unstable information. We therefore examined each factor separately.
For every factor, the sign of its training-period mean daily Rank IC determined the trading direction. We kept that direction fixed in the holdout, buying the top quartile and shorting the bottom quartile—five stocks on each side in this universe. Boundary ties shared weight equally, without consulting future returns.
We calculated each five-session long-minus-short return and regressed it on the corresponding QQQ return:
Five-session long–short return = alpha + beta × QQQ five-session return + error
Here, alpha is a five-session regression intercept. The portfolio has 100% long exposure and 100% short exposure, or 200% gross exposure. Scaling it to 100% gross would halve both returns and alpha. These figures exclude trading, borrowing, and financing costs, and overlapping portfolio returns cannot simply be compounded each day into an equity curve.
QQQ was the selected market proxy. This was not alpha after controlling for every risk factor, nor a full CAPM or Fama–French specification using a risk-free rate.
We used Newey–West/HAC inference with four lags to account for serial dependence, checked seven lags and block resampling, and applied Holm correction across the ten factor tests. The covariance method is documented in Statsmodels.
| Factor | Long side fixed from training | Holdout alpha / five sessions | Holm-adjusted p |
|---|---|---|---|
| News count | High values | −0.30% | 0.4431 |
| Scored count | High values | −0.71% | 0.1560 |
| Mean JEV score | Low values | +2.30% | 0.0025 |
| Positive-news share | High values | −1.62% | 0.0030 |
| Negative-news share | High values | +2.08% | 0.0039 |
| Materiality | High values | +2.83% | 0.0447 |
| Relevance | Low values | −2.00% | 0.1560 |
| Uncertainty | High values | +2.28% | 0.0851 |
| Confidence | Low values | +0.71% | 0.2637 |
| Decayed event impact | High values | −1.23% | 0.0758 |
Swipe to view all columns →
Among the positive estimates, materiality, reversed mean JEV score, and negative-news share passed the Holm-adjusted test in this sample. A negative alpha could also be statistically significant, as with positive-news share. That meant the training-selected direction failed; it was not a reason to reverse the direction after seeing the holdout and report success.
The inverse relationship also needs care. Low mean JEV scores and high negative-news shares performed better. This could be consistent with reversal or information already reflected in prices, but the experiment did not identify the mechanism. It did not establish a general rule to buy bad news.
Materiality was more consistent across periods. Its raw Rank IC was +0.067 in training and +0.222 in the holdout. After removing the cross-sectional effects of news count, five- and twenty-session momentum, and twenty-session volatility from both factor ranks and return ranks, the corresponding partial correlations were +0.092 and +0.174. Materiality describes the scale of a development, not whether it is favorable.
The crucial control: could fixed stock selection explain the returns?
An estimated five-session alpha above 2% calls for a closer look at its source.
Some stocks receive more news persistently. Others tend to receive high materiality scores. If a factor keeps selecting the same stocks, its portfolio could perform well because those stocks happened to do well during this period, even if daily news updates added little information.
We added a post-hoc diagnostic: replace each stock’s changing factor with its training-period mean, freeze the resulting ranking for the entire holdout, and compare that portfolio with the one using refreshed daily scores. Both used training-selected directions. The fixed portfolio did not use holdout returns to select stocks.
| Factor | Mean spread: refreshed / fixed | Incremental alpha (Holm p) |
|---|---|---|
| Materiality | +2.84% / +2.27% | +0.60 pp (0.618) |
| Reversed mean JEV | +2.28% / +2.98% | −0.66 pp (1.000) |
| Negative-news share | +2.02% / +2.54% | −0.48 pp (0.190) |
| Uncertainty | +2.28% / +3.21% | −1.02 pp (1.000) |
Swipe to view all columns →
The middle column shows mean five-session long–short spreads using refreshed scores, then fixed rankings. The final column gives the QQQ-adjusted regression intercept of their paired difference and its Holm-adjusted p value. The intercept need not equal the direct subtraction of the two spreads. “pp” means percentage points.
No factor achieved a positive increment that passed Holm correction in this control. Materiality’s estimated increment was positive but uncertain. For reversed mean JEV and negative-news share, the fixed portfolios performed better.
That did not prove that JEV scores were useless. It meant the available data could not establish how much of the result came from refreshed event information rather than persistent stock exposure.
Concentration: how much came from META?
We also removed one stock at a time, retaining the original training directions and rebuilding quartile weights across the remaining universe.
META contributed 43.8%, 54.5%, and 56.6% of the net long–short return spreads for materiality, reversed mean JEV, and negative-news share, respectively. Excluding META reduced their alpha estimates to +1.61%, +0.95%, and +0.77%.
Those estimates remained positive. We did not interpret that as evidence of continued statistical significance after deletion. The fact that one stock explained so much of the spread was a reason to seek broader evidence.
There was also substantial redundancy. The mean daily rank correlation between news count and scored count was 0.984. Mean JEV score and negative-news share had a correlation of −0.795. Reversed mean JEV and negative-news share therefore conveyed much of the same information; they were not two independent alpha discoveries.
Why this is not strict point-in-time performance
The implementation explicitly distinguished what was available at the decision time. Tiingo provides both a publication timestamp and a crawl/database timestamp. An early publication timestamp does not mean that the API had already received the article. The distinction is explained in the Tiingo News documentation.
The research replay used the later timestamp plus one minute to simulate provider availability. The canonical scoring pipeline separately preserved actual receipt, scoring-completion, and availability timestamps. Scores produced today were not backdated in that pipeline.
That addresses only part of the problem.
The articles were historical, but JEV rescored them using the model available when we ran the study. An instruction to ignore later outcomes cannot establish that the model’s knowledge, revised text, and inference environment match those available at the historical decision time. Pinning a model ID supports reproducibility; it does not eliminate historical knowledge leakage.
The holdout returns had also appeared in earlier factor exploration. This was a chronological holdout for model fitting, not a completely untouched research test set. The fixed-stock control was added after concentration was observed and must be interpreted as a post-hoc diagnostic.
The experiment is therefore a retrospective study. Stronger evidence would come from continuously recording news as received, scores as generated, and predictions as issued, then waiting for the corresponding returns to mature.
What the experiment changed in our view
The absence of stable model improvement is compatible with positive individual-factor return relationships. The classifier targeted absolute gains above 5%, while the factor portfolios used continuous returns and long–short spreads. Adding every event feature also introduced redundant and unstable inputs. There were only eighteen positive classification examples in the holdout, further limiting inference.
On this evidence, materiality is the strongest candidate for another independent test. Its direction was consistent across training and holdout, its partial correlation remained positive after the selected controls, and its estimated increment over fixed stock selection was positive. Reversed mean JEV remains an auxiliary hypothesis, with negative-news share treated as a closely related alternative rather than independent confirmation.
The next study needs a longer, previously unseen period, a broader universe, and controls for industry and size exposure. Scoring more articles within these same fourteen holdout sessions might reduce sampling noise, but it would not add independent market history.
The engineering result was a working news-to-score-to-factor research pipeline. The research result was a narrower question: Can event materiality add value beyond persistent stock exposure in newly observed, genuinely point-in-time data?
That question is worth testing. The present experiment has not yet answered it well enough to support live deployment.
Experiment records and limitations
All figures use the original frozen results. The source research package contains aggregate CSV results and a figure-generation script, without article text, API credentials, or model files. Those local research files are not hosted on this article page.
The recorded validation included 55 tests for the model comparison and 22 relevant tests for the alpha analysis, independent verification of 960 return labels, and identical predictions after reloading all eighteen research models. These checks support implementation and data consistency; they do not validate future profitability.
This was a study of twenty fixed surviving stocks with a technical-feature baseline. It was not a rerun of the complete production Sticky7 strategy, and this experiment did not enable event factors in production training. The estimates and intervals should be read in the context of the short sample, overlapping returns, current-model historical rescoring, missing industry/size/value controls, and prior exploration.
References: