Fig. 3: Effect of model size and inference type on performance.

Model size and inference type influence key model metrics on the synthetic test set. Each point reflects a T5 model with identical pre-training, MC-WSRS fine-tuning, and evaluation on a synthetic dataset of medical snippets. Detection recall decreases as the model size increases. However, performance is substantially and statistically improved with inference chaining techniques such as iterative inference and elicitive inference. The inference types and model sizes do not significantly affect detection precision (percentage of the text identified as abbreviations that are actually abbreviations), and model size improves expansion accuracy (percentage of expansions with clinically equivalent meanings). Total accuracy, which we define as detection recall multiplied by expansion accuracy, is highest for the model with the most parameters with elicitive inference. n = 400 bootstrap samples of the 302 synthetic snippets, each of which contains a different collection of abbreviations. Point estimates from the original sample and 95% confidence intervals were calculated using reporting the 2.5 and 97.5 percentile values for each metric across the samples.