How the AI works, and where it can be wrong

1. What "AI" means here

Every post we scrape passes through an LLM-based analyzer. The analyzer's job is to answer three questions about each post:

• Is this post making a trading call at all, or is it commentary, a chart, a meme, news?
• If it is a call, what ticker (or tickers) is it about?
• What direction — BUY/long or SELL/short — and how confident does the caller sound?

Those structured outputs are then stored in our database and used to compute everything downstream. If the model misreads a post, the error propagates through every one of those numbers.

2. Which models we use

SignalSnitch sends post text to multiple LLMs, including Anthropic Claude and OpenAI GPT. We send only the public post text — never your account data, your watchlist, your billing info, or any other private state.

3. Failure modes we've actually seen

3.1 Ticker hallucination

The model invents a ticker that doesn't appear anywhere in the source post. We run a verification step that drops signals whose ticker doesn't literally appear in the source text, which catches the common cases.

3.2 Sentiment misread

The model flips BUY/SELL on a post that uses irony, hedging, or a confusing structure. We rely on the consensus filter (≥2 callers agreeing) to dilute the impact of a single misread.

3.3 Non-call classified as a call

Commentary or a screenshot caption that mentions a ticker gets promoted to a real BUY/SELL signal. We use the "Non-Calls" chip on the influencer detail page so you can audit what the classifier did and didn't flag.

3.4 Time-zone / date misread

The model can occasionally misread the posting time relative to market open, which affects the "entry price" we capture.

4. What we measure

A trader's hit rate is based on what actually happened in the market after each of their calls — not on how confident the AI thought the caller sounded. We take the price at the moment of the call and set two levels around it: a profit target above, a stop below. How far away those sit is scaled to how volatile that ticker already is, so a sleepy large-cap and a jumpy small-cap aren't measured against the same distance. Then we watch the next seven days of trading. Reach the target first and the call is a hit. Reach the stop first and it's a stop. A SELL call is the mirror image: the target sits below the entry price and the stop above it.

Some calls reach neither level, and those are left out of the rate entirely. Roughly one in six sit between the two lines for the whole seven days. We don't count them against the caller — a call that never resolved is not a loss. So a hit rate is computed over hits and stops only, which is why those two counts don't add up to the caller's total number of calls.

Sample size matters more than the headline number. We always show the number of calls next to every percentage so you can weigh it yourself.

5. Safeguards we run

We filter out signals where the AI invented a ticker that wasn't actually in the post. We don't treat a single trader's opinion as a "consensus" — several independent callers have to agree first. Hits and stops are settled against real price data after the fact, and a call that reached neither level is dropped from the rate rather than scored as a loss. And we periodically re-check older signals as our analysis improves.

6. What you should do

Click through to the source post. Every signal card links to the original tweet / Reddit thread / StockTwits message. Read the post yourself before acting on any signal.

Treat numbers as ranges, not points. Every hit rate carries uncertainty from sample size, analyzer error, and survivorship. It also has a floor nowhere near 50%: the profit target sits closer to the entry price than the stop does, so a caller picking at random still lands near 61.5%. Read every hit rate against that number, not against half. Our strongest deciles sit around 68%, so the gap between lucky and good is narrower than it looks.

Report mistakes you spot. Email support@signalsnitch.io with the signal's URL. We use those reports to refine the analyzer prompt.

7. Bottom line

SignalSnitch is a research tool that uses imperfect AI on imperfect input (social media) to produce imperfect output. The pipeline is good enough to surface patterns and study crowd psychology — which is the use case we promise. It is not good enough to be a trading signal you act on without your own verification.