What all are the contexts where it is okay to use OldStyle numerals? What has been the history of these numerals in and outside the TeX world?
OldStyle numerals are not really appropriate for math. Some reasons:
1) some shapes are effectively indistinguishable from letters, e.g. OldStyle zero is very close to the shape of the letter lowercase "oh";
2) the variability of the height/depth of the OldStyle numerals could be confusing in an environment with mixed sub- and superscripts.
That said, I think I've seen OldStyle numerals used in chemical formulas, where the vocabulary is both more limited and well defined.
OldStyle numerals have been used effectively in all-numeric tables, in particular ephemerides, although I think these are no longer common; a copy on paper would likely be dated before World War I. (Any numerals intended for use in tables must be uniform in width, so that columns will line up neatly. The numerals in Computer Modern are all of this design. A more comprehensive typeface can include four styles: tabular and text (variable width) "lining" digits, and tabular and text OldStyle digits.)
Acceptable (and accepted) uses in text are in dates and page numbers (both for the page itself and for cross references), where there is almost no possibility of ambiguity. This applies as well to numbered items: sectioning, list items, theorems, equation numbers. It's wise to be very careful to mark even trivial math appropriately, to distinguish.
Although OldStyle numerals may be available in monospace ("typewriter") fonts, that style, since it is already so constrained, doesn't really lend itself to their use.
Donald Knuth wrote an article for TUGboat, Typesetting Concrete Mathematics, https://tug.org/TUGboat/tb10-1/tb23knut.pdf , that touches on the reasons for these distinctions.