Transformer encoder-decoder models (seq2seq) transformer huggingface-transformers transformer-encoder-decoder seq2seq
- challenge
- GLUE-LM-GAP
- submitter
- kaczla
- submitted
- 2023-03-06 15:48:25.968755 UTC
| # |
model_name |
dev-0 PerplexityHashed |
test-A PerplexityHashed |
|
|---|
| 152 |
LongT5-Local-base |
1006246.726444
|
1008375.972086
|
|
| 130 |
ByT5-base |
854668.733905
|
855967.364657
|
|
| 125 |
ByT5-small |
577506.468555
|
584584.351424
|
|
| 124 |
FLAN-T5-large |
544241.485156
|
562529.862349
|
|
| 123 |
T5-efficient-mini |
493098.539241
|
510519.379968
|
|
| 117 |
T5-efficient-tiny |
408282.416232
|
418979.060903
|
|
| 116 |
T5-large-v1_1 |
420124.898821
|
415089.099066
|
|
| 112 |
FLAN-T5-small |
306911.104213
|
324604.408047
|
|
| 109 |
T5-large-v1_1-lm-adapt |
337413.972666
|
300663.100084
|
|
| 108 |
T5-small-v1_1 |
290626.365308
|
282303.546243
|
|
| 107 |
T5-efficient-small |
303742.494167
|
277887.449243
|
|
| 106 |
T5-base-v1_1 |
290005.373172
|
267972.646440
|
|
| 102 |
T5-small-v1_1-lm-adapt |
239393.520998
|
233990.717024
|
|
| 101 |
LongT5-TGlobal-base |
199552.610743
|
211414.917005
|
|
| 96 |
T5-efficient-base |
152750.729495
|
124430.988625
|
|
| 95 |
FLAN-T5-base |
103892.908235
|
118104.568290
|
|
| 94 |
T5-efficient-large |
105072.298767
|
84932.099781
|
|
| 92 |
T5-base-v1_1-lm-adapt |
74652.507386
|
63737.130182
|
|
| 61 |
mT5-small |
4691.781349
|
3976.321841
|
|
| 55 |
mT5-base |
3046.209545
|
2491.979920
|
|
| 46 |
mT5-large |
2055.165101
|
1615.687190
|
|
| 29 |
T5-small |
995.334314
|
782.865991
|
|
| 18 |
Switch-base-8 |
425.605103
|
349.968606
|
|
| 16 |
T5-base |
358.614059
|
274.694340
|
|
| 13 |
T5-large |
253.863088
|
192.323406
|
|
| Parameter |
Value |
|---|
| depth |
1 |
| top_k |
15 |