Sorry, I really don’t care to continue talking about the difference between supervised and unsupervised learning. It’s a pattern used to describe how you are doing ML. It’s not a property of a dataset (you wouldn’t call Dataset A “unsupervised”). Read the Wikipedia articles for more details.

No, in that case there’s no labelling required. That would be unsupervised learning.

https://en.wikipedia.org/wiki/Unsupervised_learning

Conceptually, unsupervised learning divides into the aspects of data, training, algorithm, and downstream applications. Typically, the dataset is harvested cheaply “in the wild”, such as massive text corpus obtained by web crawling, with only minor filtering (such as Common Crawl). This compares favorably to supervised learning, where the dataset (such as the ImageNet1000) is typically constructed manually, which is much more expensive.

Ground truth labels are just prescriptive labels that we recognize as being true. The main thing that distinguishes unsupervised from supervised is that in unsupervised learning, what is “good” is learned from the unstructured data itself. In supervised learning, what is “good” is learned from some external input, like “good” human-provided examples.

No, it’s unsupervised. In pre-training, the text data isn’t structured at all. It’s books, documents, online sources, all put together.

Supervised learning uses data with “ground truth” labels.

This pre-training was done by Meta. It’s what Llama-3.1-405B is (in contrast to Llama-3.1-405B-Instruct). https://huggingface.co/meta-llama/Llama-3.1-405B

Training Data

Overview: Llama 3.1 was pretrained on ~15 trillion tokens of data from publicly available sources. The fine-tuning data includes publicly available instruction datasets, as well as over 25M synthetically generated examples.

Unsupervised training happens during the pre-training phase when you dump all kinds of quality documents and it learns the relationship between tokens

The article you linked to uses SFT (supervised fine tuning, a specific training technique) as its alignment strategy. There are other ways to fine-tune a model.

I guess I’m wondering if you can train on these partial responses without needing the full rest of the output, without the stop token, or if you need full examples as the article hints to.

Can SFT be used on partial generations? What I mean by a “steer” is a correction to only a portion, and not even the end, of model output.

For example, a “bad” partial output might be:

<assistant> Here are four examples:
1. High-quality example 1
2. Low-quality example 2

and the “steer” might be:

<assistant> Here are four examples:
1. High-quality example 1
2. High-quality example 2

but the full response will eventually be:

<assistant> Here are four examples:
1. High-quality example 1
2. High-quality example 2
3. High-quality example 3
4. High-quality example 4

The corrections don’t include the full output.

Thank you so much, that exactly answers my question with the official response (that guy works at Meta) that confirms it’s the same base model!

I was concerned primarily because in the release notes it strangely didn’t mention it anywhere, and I thought it would have been important enough to mention.