浏览代码

Add GLM-Large-Generation model

duzx16 2 年之前
父节点
当前提交
43b27f8711
共有 1 个文件被更改,包括 13 次插入0 次删除
  1. 13 0
      configs/model_glm_large_generation.sh

+ 13 - 0
configs/model_glm_large_generation.sh

@@ -0,0 +1,13 @@
+MODEL_TYPE="glm-large-generation"
+CHECKPOINT_PATH="/zhangpai21/checkpoints/glm-large-en-generation"
+MP_SIZE=1
+MODEL_ARGS="--model-parallel-size ${MP_SIZE} \
+            --vocab 30592 \
+            --num-layers 24 \
+            --hidden-size 1024 \
+            --num-attention-heads 16 \
+            --max-sequence-length 513 \
+            --tokenizer-type glm_BertWordPieceTokenizer \
+            --tokenizer-model-type bert-large-uncased \
+            --task-mask \
+            --load ${CHECKPOINT_PATH}"