The NASA Task Load Index appeared in roughly one in ten CHI papers last year. It has become close to a default instrument, which is exactly why we should ask whether we are all using it the same way. We read 522 CHI papers from 2006 to 2024 to find out. The short answer: the instrument itself is fine; the problems lie in how we implement it. Our paper is now accepted at ACM TOCHI.
Six things to do differently, in the order you would encounter them when running a study: check that workload is what you actually want to measure; explain every subscale to your participants; watch the Performance subscale (higher means worse); use the original 21-point scale and report on a 0-to-100 range; if you have no comparison condition, run the pairwise comparisons; and if you have to shorten the instrument, use Short-TLX (Mental Demand, Physical Demand, Performance).
Five papers used the NASA-TLX at CHI 2006; 99 used it in 2024. Across the whole period it appears in 522 papers, or 5.41% of everything CHI published. Only 17 of the 522 papers ran the pairwise weighting step. 340 never said how many divisions their rating scale had. 79 reported no statistical test at all. We pulled subscale values from 683 tasks in 185 papers and reanalyzed them the way Hart and Staveland analyzed their original data: with the original fine-grained scale, Frustration loads onto the cognitive-affective factor; with 5- and 7-point scales, it loads onto the physical factor instead. Same construct, coarser instrument, wrong structure.
Guidelines are easier to follow when something implements them for you, so we built one. The response collection app (sebaram.github.io/nasa-tlx-tools) uses the 21-point scale by default, highlights the Performance endpoints to prevent the reversal error, lets you switch weighting off or on, exports CSV and is open source. Separately, there is an interactive database of all 522 papers (sebaram.github.io/nasa-tlx), filterable by year, device category, participant count, and analysis method.
Links: NASA-TLX response collection app, Interactive database of 522 CHI papers.
Era: TU Clausthal, Human-Centered Information Systems (since April 2026).
Source: kaikunze.de (16 August 2026).