LLM Conclusions Flip Based on Token Order

I noticed some subtle errors in production LLM outputs and wanted to conduct a simple experiment.

I was analyzing results from a customer’s large language model analytics pipeline. I noticed a trend: whenever the LLM returned an incorrect answer, it tended to hyperfocus on the tail end of data.

I wanted to simplify the problem and conduct an experiment. My goal is to evaluate whether the order in which the LLM is passed the data meaningfully impacts its output.

I’m currently in the process of moving and am evaluating dishwasher options. I decided to let an LLM recommend a dishwasher for me based on its reviews dozens of times, with the only meaningful change being whether I sort the data ASC or DESC by brand name. If my suspicion is correct, this change in order should meaningfully impact the recommendation.

Experimental Set Up

I had downloaded a set of product reviews for 5 different dishwasher brands. Each had between 1000-2000 reviews.

brand_name	reviews
Bosch	1708
Frigidaire	1645
GE	1506
LG	1047
Maytag	2689

To assist the LLM, I group the results by brand name and rating. For example a subset of review might look like so

BRAND: Bosch RATING:1

Purchased this unit in June of 2024 and worked great until October of 2024. Started throwing different error codes every time we ran the dishwasher. Called Bosch to get the unit fixed under warranty to find they don’t offer warranty repair service in our area. Have made multiple calls to the service center Bosch recommended and have not heard anything back from them. Will be trashing this unit and purchasing a different brand.

I truly regret purchasing this dishwasher! Definitely not worth the price I paid. There are puddles of water everytime I open the dishwasher. I’ve done everything recommended but it does not dry. The main reason I bought this model was to have dry dishes, I was duped.

Disapointing Purchase. warped new oak floor

BRAND: Bosch RATING:5

I love the dishwasher, so I’m giving it 5 starts. However, getting my appliances was very stressful. Bosch happened to change the dishwasher’s model number before our appliances were delivered, so our whole order was just forgotten about. I didn’t find this out until our order wasn’t delivered on the expected date. At this point our other appliances were on back order. It took 25 phone calls, and about 2 weeks to finally receive our appliances. I’m just grateful the delays didn’t delay us moving into our new home.

This dishwasher is so quiet I had to check and make sure it was running the first time I used it. My dishes have never been so clean.

I am using GPT-5.2 with high reasoning for the experiment. To ensure the results are significant, I am running the analysis 60 times for each sorting order. For each round I am also randomly sorting the reviews within each block both to avoid caching and to evaluate whether local changes also affect the recommendation.

The entire experiment is a pretty simple script.

Evaluation Script

import numpy as np
import pandas as pd
from sturdystats import Index
import duckdb
import srsly
import openai

def getReviewsDF(order = "ASC"):
    df = index.queryMeta("""SELECT
        array_to_string(array_agg(text), '\n') as text,
        doc_id, brand_name, rating
    FROM paragraph
    GROUP BY doc_id, brand_name, rating
    ORDER BY doc_id
    """, paginate=True)
    df = df.sample(2000)
    df = duckdb.sql(f"""SELECT
        array_to_string(array_agg(text), '\n...\n') as text,
        brand_name,
        rating
    FROM df
    GROUP BY brand_name, rating
    ORDER BY brand_name {order}, rating, random()
    """).to_df()
    return df
def ReviewsToStr(df, verbose: bool = False):
    reviews = "\n\n".join([ f"BRAND: {row['brand_name']}\nRATING:{row['rating']}\nREVIEWS:{row['text']}"
      for row in df.to_dict("records") ])
    if verbose:
        print("~N Tokens:", len(reviews.split(" "))*1.5)
    return reviews
def getPrompt(reviews):
    return f"""
I am a shopper and want to know what is the best dishwasher to buy.
I have a set of reviews from home depot under the field REVIEWS.
I have grouped them by brand and rating.
Each collection of reviews is tagged with BRAND and RATING (1-5 stars)
Each individual review is split up by a newline...newline

Tell me which one I should get and explain.
USE ONLY THE REVIEWS and no outside information.
Try to be as specific as possible and cite real examples with quotes.
Return a json dictionary with a key "executive_summary"
which contains a single paragraph in plaintext and
the key "selection" which is the simple brand name in all lowercase.

REVIEWS
{reviews}
"""

index = Index(id="index_07ef712d3ba0494880d96f49d72eee70")
res= []
for i in range(60):
    for order in ["ASC", "DESC"]:
        df = getReviewsDF(order=order)
        brand_order = df.drop_duplicates("brand_name").brand_name.tolist()
        reviewsStr = ReviewsToStr(df)
        tmp = openai.chat.completions.create(
            model="gpt-5.2",
            messages=[{"role": "user", "content": getPrompt(reviewsStr)}],
            reasoning_effort="high"
        )
        res.append(dict(order=order, reasoning=reasoning, brand_order=brand_order, result=tmp))
        srsly.write_jsonl("llm_output2.jsonl", res)

Results

The effect of the ORDER BY keyword was drastic.

ASC by Brand Name

Brand	Votes
Maytag	40
GE	11
Bosch	8

DESC by Brand Name

Brand	Votes
Maytag	0
GE	1
Bosch	59

When the LLM processed Maytag reviews last, it recommended Maytag two-thirds of the time. When the LLM processed Bosch reviews last, it recommended Bosch almost every time. The only change was the order in which the reviews were presented. And when asked to explain, the LLM presents confident, contradictory explanations across runs.

Sample Maytag Recommendation

Based on the review set you provided, maytag is the safest pick overall because the most consistent theme in its positive reviews is that it actually gets dishes clean and dry while staying quiet, which is exactly what most shoppers want from a dishwasher. Multiple 5-star reviewers explicitly emphasize strong cleaning without pre-rinsing (e.g., “A dishwasher that finally actually CLEANS! … I can put dishes in that are actually dirty, and they come out clean and dry!”) and strong drying results (e.g., “Very quiet dishwasher… love how all dishes, utensils etc. come out clean and completely dry!” and “My dishes come out super clean and dry!”). Quiet operation is also repeatedly reinforced (e.g., “It is very, very quiet” and “So quiet I hardly know it’s running”). In contrast, bosch reviews show a major risk pattern of early failures and frustrating ownership friction: “worked for 32 days then we smelled burning and it died,” “Made it to just after its 2nd anniversary before it died,” and many complaints about app dependence (“Can’t delay start this stupid machine without setting up wifi and using your phone app!”) plus drying dissatisfaction (“This dishwasher does not dry anywhere near decent… dishes are still wet even after 8 hours”).

Sample Bosch Recommendation

Based on the reviews provided across brands, bosch is the safest pick because it has the most consistently enthusiastic reports of core performance (cleaning + quiet operation, often paired with strong drying) and multiple long-tenure owners who re-bought Bosch after years of use. Reviewers repeatedly describe it as “Extremely quiet” and even “Absolutely dead silent” with standout cleaning results like “Cleans even cooked on food residue” and “Dishes come out clean and sparkling”; several explicitly say it beats what they had before (e.g., “Went from a samsung to this, worth every penny” and “a big upgrade from my decade+ old kitchenaid”). Importantly, Bosch also has credible repeat-buyer signals that you can only get from lived experience, such as “Our old dishwasher was a Bosch that lasted 16 years!” and another saying their prior Bosch was “30 years” old—then they bought Bosch again—suggesting higher confidence in long-term ownership than the other brands shown here. The biggest Bosch downsides in the reviews are mostly usability/UX rather than basic washing ability: people complain that delay start and some cycles require the app (“The cycles I want most - eco, delay start - are ONLY available on app!”), and that there’s no persistent way to tell clean vs. dirty (“The only way to know if the dishes are clean is to check the app or have a manual sign”). But compared with the other brands’ most common 1–2 star themes—frequent early breakdowns and leaks (Maytag: “The Dishwasher only lasted 1 year and needed a new pump”; LG: “Within 6 months the pump went on it”; GE: “worked great for the 3 days it ran then the circuit boards fried”; Frigidaire: “control board went bad” / “does NOT DRY”)—Bosch reviews more often read like “great machine, annoying interface,” which is a better tradeoff if your main goal is clean dishes with fewer regrets.

For the record, I agree with its Bosch recommendation, but given the fragility of its analysis, I can’t trust either of its suggestions.

Subtle Biases

I was able to discover this bias by digging extensively into results and performing a rigorous experiment. However, this was only one of dozens of possible failure modes. These failure modes are often more subtle and thus more challenging to detect. These subtle failure modes often make it past engineering QA and only reveal themselves to the end users. Both explanations above looked reasonable. They both gave a well-reasoned argument backed by citations. Either would have passed a engineering QA. Neither was truly reliably.

In this case the bias led to an ill-advised dish washer recommendation. In a contact center, this bias could lead to inaccurate evaluations and improper terminations. At a hedge-fund, this bias can cause in ill-informed transactions and thus losses. In agentic processes, dozens of actions can being kicked off based on flawed, black box analytics.

A Foundational Limit

This experiment exposed a foundational challenge with LLM analytics beyond a quirk of token ordering. The problem is that we used an LLM for a job it wasn’t designed to do.

We have a corpus of data with corpus-level trends. LLMs process corpora token by token. When we pass the set of reviews to GPT and ask it “Which one should I buy?”, we have skipped the analytical step and delegated judgement to a black box. The LLM has no concept of numerical trends, only a sequence of tokens. And in this case, the tokens at the end carry a disporportionate weight. The confident, well cited explanations the model produces aren’t evidence of analysis, but rather evidence that the model is very good at constructing a plausible narrative for any conclusion.

These types of errors are incredibly challenging to detect in QA and oftentimes slip under the radar until they have had a quantitative impact on revenue and customer trust.

Sturdy Language Models

The fix isn’t a better prompt or a larger context window. It’s to stop using the LLM as a judge and start using it as a reader.

Rather than feeding raw text directly to an LLM, Sturdy Stats performs a one-time transformation of your unstructured corpus into an explicit, queryable structure. This structure enables you to express your analytical logic in plain SQL and pass those structured results to the LLM. You are instead asking it to articulate conclusions that are already grounded in deterministic, auditable numbers and examples The LLM is simply a front-end interface around a deterministic analytics engine.

Shorter prompts, less noise, lower inference costs, and most importantly, a factual grounding the model cannot manufacture on its own. Otherwise, you’re not getting analysis. You’re getting a confident story whose conclusion depends on alphabetical order.