Articles

How we built a Large Response Model for Hong Kong

Date
Category
Research

In February 2025, we introduced Dlab-852-Mini, Decisions Lab's first Large Response Model. We built it to simulate how people might respond to decisions and social situations in Hong Kong. We used CDEval to measure the base model’s cultural bias and check whether fine-tuning brought its responses closer to a Hong Kong cultural reference.

Why we started in Hong Kong

Imagine asking a large language model how people would respond to a new workplace policy. Its answers depend on what it assumes about how people handle uncertainty or disagree with a manager. Those judgments can vary within an audience and across cultures.

When we started simulating responses in Hong Kong, the base model’s answers could sound plausible while consistently favoring particular cultural norms. To measure that bias, we ran Microsoft’s Phi-3.5-mini-instruct on CDEval and compared its scores with the Hong Kong reference. That gave us a baseline for judging what fine-tuning changed.

Hong Kong vs. Microsoft Phi-3.5-mini-instruct

  • Power distance

    Hong Kong68
    Microsoft43.7
  • Individualism

    Hong Kong25
    Microsoft60.4
  • Uncertainty avoidance

    Hong Kong29
    Microsoft60.6
  • Masculinity

    Hong Kong57
    Microsoft37.3
  • Long-term orientation

    Hong Kong61
    Microsoft77.7
  • Indulgence

    Hong Kong17
    Microsoft66.4
Hong Kong’s Hofstede reference scores and the base model’s average scores on English-language CDEval items, shown on a 0–100 scale.

Building Dlab-852-Mini

We started with Microsoft’s Phi-3.5-mini-instruct and fine-tuned it on a dataset we curated for Hong Kong. We could then compare the original model with Dlab-852-Mini, our adaptation for response simulation.

We use Large Response Model to describe a language model adapted to simulate how a defined audience might respond to a scenario. For Dlab-852-Mini, that meant evaluating whether fine-tuning shifted patterns in the model’s answers toward a Hong Kong cultural reference.

Measuring cultural bias with CDEval

CDEval was developed by researchers affiliated with Beijing Jiaotong University, Microsoft Research Asia, and Peng Cheng Lab. Its questions ask models to choose between responses that reflect different cultural orientations. The vendor example below illustrates how a question can reveal a preference about uncertainty.

Illustrative example

A team needs to choose a vendor by Friday. One vendor has a proven track record. The other is cheaper, but its delivery date is uncertain. Which would you choose?

“I’d choose the proven vendor. If delivery slips, we miss our own deadline.”

Microsoft Phi

“I’d choose the cheaper vendor and keep the proven one on standby. It’s worth taking the risk if we have a fallback.”

Dlab-852-Mini

We ran Dlab-852-Mini on the same English-language prompts as the base model. Across the benchmark questions, we compared both models’ scores with the Hong Kong reference to see how much of the original gap fine-tuning had closed.

What changed after fine-tuning

We rescored the archived responses from both models using the same method. Dlab-852-Mini moved closer to the Hong Kong reference on individualism, uncertainty avoidance, and indulgence. It moved farther away on power distance, masculinity, and long-term orientation.

Closeness to the Hong Kong cultural reference

  • Power distance

    Microsoft Phi75.7
    Dlab-852-Mini69
  • Individualism

    Microsoft Phi64.6
    Dlab-852-Mini94.4
  • Uncertainty avoidance

    Microsoft Phi68.4
    Dlab-852-Mini94.5
  • Masculinity

    Microsoft Phi80.3
    Dlab-852-Mini78
  • Long-term orientation

    Microsoft Phi83.3
    Dlab-852-Mini80.5
  • Indulgence

    Microsoft Phi50.6
    Dlab-852-Mini86.4
Higher means closer. We subtract the absolute gap from 100 points, so 100 is an exact match. The chart starts at 50.

This was an English-language comparison against group-level cultural scores. A model can move closer to a group average while still missing important differences between people within that group. Evaluating how well it predicts individual choices or behavior outside the benchmark would require separate evidence.

From cultural alignment to buyer research

We’ve applied the same lessons to buyer simulations. We use evidence about your target audience to adapt the model to how those buyers weigh decisions, including the objections they raise and the proof they need.

Sources