Happy Tuesday! Every AI product decision eventually runs into the same wall: the model can be smart, but if the answer takes three seconds to arrive, users bounce. Today’s word — latency — is the one metric that shows up in almost every serving-infrastructure conversation, and it’s worth being able to talk about it precisely, not just say “it’s slow.”
📖 Word of the Day: Latency
- Word: latency
- IPA: /ˈleɪ.tən.si/
- Vietnamese meaning: độ trễ — khoảng thời gian từ lúc gửi yêu cầu đến lúc nhận được phản hồi
- Example sentences:
- “We cut inference latency from 800 milliseconds to 120 by adding a caching layer in front of the model.”
- “Users start complaining the moment latency crosses the one-second mark, even if the answer is perfect.”
- “We accepted a small drop in accuracy in exchange for lower latency, since the model runs on-device.”
Listen and practice pronunciation:
Vocabulary Table — Model Serving
| Phrase | Vietnamese | Example |
|---|---|---|
| Time to first token (TTFT) | thời gian đến token đầu tiên | ”TTFT dropped by 40% after we switched to speculative decoding.” |
| Cold start | khởi động nguội | ”Cold start on the serverless endpoint added two full seconds to the first request.” |
| Throughput | thông lượng, số lượng xử lý được trong một khoảng thời gian | ”We batched requests to increase throughput without hurting latency too much.” |
| Tail latency | độ trễ đuôi (thường tính ở mốc p99) | “Tail latency spiked because one GPU node in the cluster was overloaded.” |
| Streaming response | phản hồi trả về theo từng phần, dạng luồng | ”Streaming the response makes the wait feel shorter even when total latency is the same.” |
🗣 Pronunciation Guide
“Latency” breakdown — /ˈleɪ.tən.si/:
- Three syllables: LAY-tuhn-see
- Primary stress is on the first syllable: LAY-tuhn-see
- The middle syllable is a quick, unstressed “tuhn” — don’t stretch it out
- The ending “-cy” is a soft “see,” same sound as in “urgency” or “fluency”
Practice sentence (read aloud 3x):
“Lower latency directly improves the user’s perceived speed, even when the model itself hasn’t changed at all.”
Say it slowly the first time, focus on stressing “LAY” the second time, then read it at natural conversational speed the third time.
✍️ Exercise 1: Fill in the Blank
Fill each blank with one term: latency, cold start, throughput, tail latency, streaming response.
- “The p50 looked fine, but the _______ was the real problem — some users waited over four seconds.”
- “We switched to a _______ so the chat UI could show words as they’re generated instead of one long pause.”
- “The serverless function’s _______ meant the very first request after idle time was painfully slow.”
- “Our load test measured _______ in requests per second, not just how fast a single request came back.”
- “The team’s whole Q3 goal was reducing average _______ without sacrificing answer quality.”
Show answers
- tail latency
- streaming response
- cold start
- throughput
- latency
✍️ Exercise 2: Translate to English
Translate these Vietnamese sentences using today’s vocabulary.
- “Độ trễ tăng vọt mỗi khi có quá nhiều người dùng gọi API cùng lúc.”
- “Chúng ta đã thêm một lớp cache để giảm thời gian đến token đầu tiên.”
- “Khởi động nguội là lý do request đầu tiên sau giờ nghỉ luôn chậm hơn hẳn.”
Show answers
- “Latency spikes whenever too many users call the API at the same time.”
- “We added a caching layer to reduce time to first token.”
- “Cold start is why the first request after idle time is always noticeably slower.”
💬 Idiom of the Day: “In the blink of an eye”
- Vietnamese meaning: nhanh như chớp mắt, diễn ra cực kỳ nhanh
- Example 1: “With the new caching layer, the response comes back in the blink of an eye compared to last month.”
- Example 2: “Users won’t notice a 50ms improvement, but they will notice when a two-second wait turns into the blink of an eye.”
📺 Recommended Watching
- Two Minute Papers — short, clear breakdowns of AI research, including model efficiency and inference speed
- AI Explained — accessible deep dives on how modern models actually run in production
- YouGlish — hear “latency” pronounced in real technical talks
🎯 Daily Challenge
Next time you use an AI tool today, notice how long the response actually takes and say out loud: “That took about [X] seconds — the latency felt [fast/slow].” Naming the number and the feeling in one sentence is what makes this vocabulary usable in a real standup, not just a flashcard.