浏览代码

Set nucleus sampling as the default strategy

The original script use `num-beams=1` as the default generation strategy, which is a bad one for most LMs. We change to necleus sampling with `topp=0.7` and `temperature=1.0`, which can be much better.
Shaw 2 年之前
父节点
当前提交
4891968041
共有 1 个文件被更改,包括 4 次插入4 次删除
  1. 4 4
      scripts/generate.sh

+ 4 - 4
scripts/generate.sh

@@ -14,14 +14,14 @@ NUM_BEAMS=4
 LENGTH_PENALTY=1.0
 NO_REPEAT_NGRAM=3
 # BaseStrategy args
-TEMP=0.9
-TOPK=1
-TOPP=0
+TEMP=1.0
+TOPK=0
+TOPP=0.7
 
 ARGS="${main_dir}/generate.py \
        --seed $SEED \
        --mode inference \
-       --sampling-strategy BeamSearchStrategy \
+       --sampling-strategy BaseStrategy \
        --out-seq-length $MAX_OUTPUT_LENGTH \
        --min-gen-length $MIN_GEN_LENGTH \
        --num-beams $NUM_BEAMS \