f238861e6606ebafe93171d2a43ddd12df03fa59
Transformer encoder-decoder models (seq2seq) transformer huggingface-transformers transformer-encoder-decoder seq2seq

 

challenge
GLUE-LM-GAP
submitter
kaczla
submitted
2023-02-23 11:14:06.119968 UTC
original repo
gonito_prods://gitlab.com/kaczla/glue-lm-gap.git / branch encoder-decoder
publicly available at
ssh://git.wmi.amu.edu.plglue-lm-gap / branch submission-08001
browsable at
https://git.wmi.amu.edu.plglue-lm-gap/tree/submission-08001
clone by
git clone --single-branch ssh://git.wmi.amu.edu.plglue-lm-gap -b submission-08001
# model_name dev-0 PerplexityHashed test-A PerplexityHashed
131 FLAN-T5-small 853010.711229 860545.059404
120 T5-base-v1.1 462216.715180 456565.137625
113 T5-small-v1.1 341731.446603 361959.431007
103 FLAN-T5-base 266041.436835 266466.400683
89 T5-base 56915.639536 60987.821059
73 T5-small 10171.933229 9998.940853
Parameter Value
depth 1
top_k 15