Appending the previous turn to a follow-up query lost three of four retrieval cases

Appending the previous utterance assumes the missing noun is in the previous utterance. In the logged conversations it often was not. When a user asks "Can you explain that in more detail?", the "that" usually points at what the avatar just explained, not at the question the user just asked ^[10]. The correct answer only ranked first once the full exchange went into the query, user utterance and avatar response together ^[11].

The post, originally published in Japanese ^[24], sorts the references by what they point at. Japanese grammar separates situational deixis, which refers to something present in the speech situation, from textual deixis, which refers to a previously mentioned expression ^[7]. In a voice-plus-screen interface the situation is the screen ^[7]. "What's on this page?" resolves against the current screen and "What other articles are on this site?" against the site ^[6]. A query builder that reaches into the transcript for those two is reading the wrong store. Textual deixis then splits again, into the immediately preceding exchange and one several turns back ^[8].

The three-turns-back case is where the numbers are. "Earlier" does not reach with the immediate exchange alone, and the developer treats that vagueness as inherent in the expression ^[12]. Partial backtracking scored worse than adding nothing, dragged down by the topics in between ^[13]. Adding only the relevant exchange scored 0.813 against 0.732 for adding everything, about 11 percent higher ^[14][1]. The post does not name the metric.

Finding the relevant exchange is the good engineering here. The utterance carries its own key: "Earlier about avatar" names the topic, so scanning the conversation history for exchanges containing "avatar" is enough ^[15]. Two-gram character overlap was sufficient for the matching, with no morphological analysis and no additional embeddings ^[16].

An earlier attempt fed extracted terms back into the prompt. The extractor dropped function words at the character level, so "それをもうちょっと詳しく" came out as "うちょ詳", usable as a search key and unreadable as Japanese ^[17]. The question was already in the prompt, so the reinsertion was removed ^[17].

User feedback said that when "in detail" appears, it should always be accompanied by either a request or an intention ^[18]. Across 22 real utterances the intent held and the surface did not: a third dropped the verb through nominalization, which the developer compares to zero anaphora ^[19]. Enforcing the rule as mandatory would have discarded roughly seven of the 22 ^[19][2]. The terms "in detail" and "specifically" appeared in all 22, which is why the developer treats the terms themselves as the most reliable indicator ^[20].

Four of the 22 ran past 25 characters, the longest at 70 ^[21]. The 25-character cutoff existed to catch "short utterances that can't be searched alone", and the same cutoff decided whether an utterance counted as a follow-up request at all, so those four got neither treatment ^[22]. The developer's conclusion is that adding context and indicating follow-up are separate decisions ^[23].

For these scores to transfer, a few things have to be true of your system: a corpus small enough that dilution moves rank order, which 38 pages is ^[5]; a UI with a screen, so the situational class exists at all ^[7]; and referent classes marked clearly enough in your language that you can branch on them. Four evaluated cases and 22 logged utterances come from one developer's site ^[5][19].