準備
下記、URLのダウンロードページから、必要な項目を入力して、「Specify custom URL」を入手しておく。
前回、Llama2を動かしてみるでLlama2を動かしたが、今回はLlama3を動かしてみる。
ダウンロード
pip install llama-stack
llama model list表示されたリストから「Llama-3.2-1B-Instruct」等、ダウンロードするモデル名をMODEL_IDに指定して下記のコマンドを実行する。
llama model download --source meta --model-id MODEL_ID実行中、下記の様に表示されたら、事前に入手した「Specify custom URL」を貼り付ける。
Please provide the signed URL for model Llama-3.2-1B you received via email after visiting https://www.llama.com/llama-downloads/ (e.g., https://llama3-1.llamameta.net/*?Policy...): 実行
torch等、実行に必要なモジュールをインストールする。
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
pip install fairscale fire blobfile実行例をGithubからCloneして取得する。
git clone https://github.com/meta-llama/llama-models.gitllama-modelsのscriptsフォルダに入っているexample_chat_completion.pyを実行する。
CHECKPOINT_DIR=~/.llama/checkpoints/Meta-Llama3.2-1B-Instruct
torchrun llama-models/models/scripts/example_chat_completion.py $CHECKPOINT_DIR実行結果（例）
User: what is the recipe of mayonnaise?
> Assistant: The classic recipe of mayonnaise! Here's a simple and traditional recipe to make mayonnaise at home:
**Ingredients:**
* 2 large egg yolks
* 1 tablespoon lemon juice or vinegar (white wine, apple cider, or white distilled vinegar)
* 1/2 cup (120 ml) neutral-tasting oil, such as canola, grapeseed, or light olive oil
* Salt, to taste (optional)
**Instructions:**
1. **Separate the egg yolks**: Crack 2 large egg yolks into a medium-sized bowl.
2. **Whisk the egg yolks**: Whisk the egg yolks with a fork until they become light and frothy.
3. **Add the lemon juice or vinegar**: Whisk in 1 tablespoon of lemon juice or vinegar to help stabilize the emulsion.
4. **Slowly add the oil**: While continuously whisking the egg yolks, slowly pour in the oil in a thin, steady stream. Start with a very slow pour and gradually increase the flow as the mixture thickens.
5. **Whisk until smooth**: Continue whisking until the mixture becomes thick, creamy, and emulsified. This should take about 5-7 minutes, depending on the temperature and whisking speed.
6. **Season with salt (optional)**: If desired, add a pinch of salt to taste.
7. **Taste and adjust**: Give the mayonnaise a taste and adjust the seasoning if needed.
**Tips and Variations:**
* **Use room temperature ingredients**: This will help the emulsion form more easily.
* **Don't over-whisk**: Stop whisking once the mixture thickens and becomes smooth. Over-whisking can lead to a separated or broken mayonnaise.
* **Add flavorings**: Try adding minced garlic, chopped herbs (e.g., parsley, dill, or chives), grated ginger, or other flavorings to create different variations.
* **Use different oils**: Experiment with different oils, such as truffle oil, chili oil, or infused oils, to create unique flavor profiles.
* **Make ahead**: Mayonnaise can be made ahead and refrigerated for up to 1 week. Give it a good stir before using.
**Troubleshooting:**
* **Separation**: If the mayonnaise separates, don't worry! Simply whisk in a little more oilVSCODEで実行する場合
pip install acceleratehuggingface形式に変換するため、GithubからtransformersをCloneする。
git clone https://github.com/huggingface/transformers.githuggingface形式に変換する。
python transformers/src/transformers/models/llama/convert_llama_weights_to_hf.py \
--input_dir /mnt/e/Work/Llama3/checkpoints/Llama3.2-1B \
--model_size 1B --llama_version 3.2 \
--output_dir /mnt/e/Work/Llama3/checkpoints/Llama3.2-1B-hfコードサンプル
import transformers
import torch
model = "/mnt/e/Work/Llama3/checkpoints/Llama3.2-1B-hf"
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
sequences = pipeline(
'I have tomatoes, basil and cheese at home. What can I cook for dinner?\n',
do_sample=True,
top_k=10,
num_return_sequences=1,
truncation = True,
max_length=400,
)
for seq in sequences:
print(f"Result: {seq['generated_text']}")
