Neuro-cognitive models of word learning propose a role for sleep in consolidating new words, yet evidence for sleep-associated memory benefits outside of experimental contexts is scarce. This study compared wake- and sleep-associated memory changes in data from Memrise, a publicly available language-learning app. Memory for foreign words and phrases remained very high in accuracy across a 7-12 hour delay, and there were no differences in forgetting between wake and sleep. However, learners were quicker to arrive at the correct translation after a period of sleep compared to wake. This sleep-associated benefit was seen for words but not phrases, and could not be fully accounted for by circadian differences in completion time. As such, we demonstrate that the behavioural benefits of sleep on vocabulary can be observed in real-world language learning, and discuss the promise for combining small-scale lab studies with naturally occurring datasets to understand learning outcomes.