20210 lines
586 KiB
Markdown
20210 lines
586 KiB
Markdown
---
|
|
license: mit
|
|
model-index:
|
|
- name: KaLM-Embedding
|
|
results:
|
|
- dataset:
|
|
config: en-ext
|
|
name: MTEB AmazonCounterfactualClassification (en-ext)
|
|
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
|
|
split: test
|
|
type: mteb/amazon_counterfactual
|
|
metrics:
|
|
- type: accuracy
|
|
value: 94.35532233883059
|
|
- type: ap
|
|
value: 60.40219300665376
|
|
- type: ap_weighted
|
|
value: 60.40219300665376
|
|
- type: f1
|
|
value: 86.52001470357649
|
|
- type: f1_weighted
|
|
value: 94.65531755022661
|
|
- type: main_score
|
|
value: 94.35532233883059
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: en
|
|
name: MTEB AmazonCounterfactualClassification (en)
|
|
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
|
|
split: test
|
|
type: mteb/amazon_counterfactual
|
|
metrics:
|
|
- type: accuracy
|
|
value: 91.71641791044776
|
|
- type: ap
|
|
value: 68.4050364584575
|
|
- type: ap_weighted
|
|
value: 68.4050364584575
|
|
- type: f1
|
|
value: 87.91854774634491
|
|
- type: f1_weighted
|
|
value: 92.0430596057422
|
|
- type: main_score
|
|
value: 91.71641791044776
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB AmazonPolarityClassification
|
|
revision: e2d317d38cd51312af73b3d32a06d1a08b442046
|
|
split: test
|
|
type: mteb/amazon_polarity
|
|
metrics:
|
|
- type: accuracy
|
|
value: 96.49945000000001
|
|
- type: ap
|
|
value: 94.97348227456295
|
|
- type: ap_weighted
|
|
value: 94.97348227456295
|
|
- type: f1
|
|
value: 96.49855824500423
|
|
- type: f1_weighted
|
|
value: 96.49855824500422
|
|
- type: main_score
|
|
value: 96.49945000000001
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: en
|
|
name: MTEB AmazonReviewsClassification (en)
|
|
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
|
|
split: test
|
|
type: mteb/amazon_reviews_multi
|
|
metrics:
|
|
- type: accuracy
|
|
value: 61.242
|
|
- type: f1
|
|
value: 59.353696237560094
|
|
- type: f1_weighted
|
|
value: 59.35369623756011
|
|
- type: main_score
|
|
value: 61.242
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB ArguAna
|
|
revision: c22ab2a51041ffd869aaddef7af8d8215647e41a
|
|
split: test
|
|
type: mteb/arguana
|
|
metrics:
|
|
- type: main_score
|
|
value: 56.569
|
|
- type: map_at_1
|
|
value: 31.080999999999996
|
|
- type: map_at_10
|
|
value: 47.432
|
|
- type: map_at_100
|
|
value: 48.247
|
|
- type: map_at_1000
|
|
value: 48.251
|
|
- type: map_at_20
|
|
value: 48.114000000000004
|
|
- type: map_at_3
|
|
value: 42.425000000000004
|
|
- type: map_at_5
|
|
value: 45.128
|
|
- type: mrr_at_1
|
|
value: 31.57894736842105
|
|
- type: mrr_at_10
|
|
value: 47.6253132832081
|
|
- type: mrr_at_100
|
|
value: 48.440395388879296
|
|
- type: mrr_at_1000
|
|
value: 48.44416076630039
|
|
- type: mrr_at_20
|
|
value: 48.30706364782469
|
|
- type: mrr_at_3
|
|
value: 42.59127548601235
|
|
- type: mrr_at_5
|
|
value: 45.347321005215804
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 7.110790588301176
|
|
- type: nauc_map_at_1000_max
|
|
value: -12.892696039828866
|
|
- type: nauc_map_at_1000_std
|
|
value: -15.5709273320573
|
|
- type: nauc_map_at_100_diff1
|
|
value: 7.117551663882657
|
|
- type: nauc_map_at_100_max
|
|
value: -12.882680977142957
|
|
- type: nauc_map_at_100_std
|
|
value: -15.56350483617667
|
|
- type: nauc_map_at_10_diff1
|
|
value: 6.903272993199564
|
|
- type: nauc_map_at_10_max
|
|
value: -13.012877497725961
|
|
- type: nauc_map_at_10_std
|
|
value: -15.947400478856006
|
|
- type: nauc_map_at_1_diff1
|
|
value: 10.03503740028087
|
|
- type: nauc_map_at_1_max
|
|
value: -13.351553937797
|
|
- type: nauc_map_at_1_std
|
|
value: -14.137614923859612
|
|
- type: nauc_map_at_20_diff1
|
|
value: 7.01754882034529
|
|
- type: nauc_map_at_20_max
|
|
value: -12.864438636302197
|
|
- type: nauc_map_at_20_std
|
|
value: -15.541510619190976
|
|
- type: nauc_map_at_3_diff1
|
|
value: 7.018587254951812
|
|
- type: nauc_map_at_3_max
|
|
value: -13.38420244471981
|
|
- type: nauc_map_at_3_std
|
|
value: -16.127099270987785
|
|
- type: nauc_map_at_5_diff1
|
|
value: 6.920961668066123
|
|
- type: nauc_map_at_5_max
|
|
value: -13.169892625713931
|
|
- type: nauc_map_at_5_std
|
|
value: -16.21272880801226
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 5.5525831294754004
|
|
- type: nauc_mrr_at_1000_max
|
|
value: -12.98089269414052
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -15.396489593627944
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 5.559525360367539
|
|
- type: nauc_mrr_at_100_max
|
|
value: -12.970885236428334
|
|
- type: nauc_mrr_at_100_std
|
|
value: -15.389102542398783
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 5.38828048977972
|
|
- type: nauc_mrr_at_10_max
|
|
value: -13.096637253890634
|
|
- type: nauc_mrr_at_10_std
|
|
value: -15.775810422484374
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 8.58091801149426
|
|
- type: nauc_mrr_at_1_max
|
|
value: -12.352949021555306
|
|
- type: nauc_mrr_at_1_std
|
|
value: -13.545487974417847
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 5.4666282281067735
|
|
- type: nauc_mrr_at_20_max
|
|
value: -12.952039027828944
|
|
- type: nauc_mrr_at_20_std
|
|
value: -15.367907454271231
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 5.1862331302405735
|
|
- type: nauc_mrr_at_3_max
|
|
value: -13.816401285559108
|
|
- type: nauc_mrr_at_3_std
|
|
value: -15.872101319770382
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 5.471097057115419
|
|
- type: nauc_mrr_at_5_max
|
|
value: -13.269134531334442
|
|
- type: nauc_mrr_at_5_std
|
|
value: -15.95735511276538
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 6.8032235432235275
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: -12.52617810408163
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -15.38677998208727
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 6.971743190062509
|
|
- type: nauc_ndcg_at_100_max
|
|
value: -12.284060222136334
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -15.203583619739097
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 5.9423315360857005
|
|
- type: nauc_ndcg_at_10_max
|
|
value: -12.649746010742199
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -16.72153869758235
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 10.03503740028087
|
|
- type: nauc_ndcg_at_1_max
|
|
value: -13.351553937797
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -14.137614923859612
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 6.379802915097805
|
|
- type: nauc_ndcg_at_20_max
|
|
value: -12.01427315352701
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -15.108250307425825
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 6.298556094258956
|
|
- type: nauc_ndcg_at_3_max
|
|
value: -13.536187803253377
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -16.999347732797407
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 6.099858591554027
|
|
- type: nauc_ndcg_at_5_max
|
|
value: -13.097631098081774
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -17.215525664264348
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -21.130247827110427
|
|
- type: nauc_precision_at_1000_max
|
|
value: 24.21748822806628
|
|
- type: nauc_precision_at_1000_std
|
|
value: 83.6578697460551
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 29.395727608507894
|
|
- type: nauc_precision_at_100_max
|
|
value: 51.676651935775695
|
|
- type: nauc_precision_at_100_std
|
|
value: 62.92260397258278
|
|
- type: nauc_precision_at_10_diff1
|
|
value: -0.25306953208178373
|
|
- type: nauc_precision_at_10_max
|
|
value: -9.710491261292093
|
|
- type: nauc_precision_at_10_std
|
|
value: -21.697648668302183
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 10.03503740028087
|
|
- type: nauc_precision_at_1_max
|
|
value: -13.351553937797
|
|
- type: nauc_precision_at_1_std
|
|
value: -14.137614923859612
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -2.084669856957687
|
|
- type: nauc_precision_at_20_max
|
|
value: 6.736841084303921
|
|
- type: nauc_precision_at_20_std
|
|
value: -0.330152716888139
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 4.202256387521114
|
|
- type: nauc_precision_at_3_max
|
|
value: -14.043068948669681
|
|
- type: nauc_precision_at_3_std
|
|
value: -19.71625712734227
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 3.2694130100522667
|
|
- type: nauc_precision_at_5_max
|
|
value: -12.7772753118202
|
|
- type: nauc_precision_at_5_std
|
|
value: -20.917228577779888
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: -21.13024782711332
|
|
- type: nauc_recall_at_1000_max
|
|
value: 24.21748822806101
|
|
- type: nauc_recall_at_1000_std
|
|
value: 83.6578697460535
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 29.395727608504448
|
|
- type: nauc_recall_at_100_max
|
|
value: 51.67665193577227
|
|
- type: nauc_recall_at_100_std
|
|
value: 62.92260397258032
|
|
- type: nauc_recall_at_10_diff1
|
|
value: -0.2530695320818313
|
|
- type: nauc_recall_at_10_max
|
|
value: -9.710491261292015
|
|
- type: nauc_recall_at_10_std
|
|
value: -21.697648668302048
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 10.03503740028087
|
|
- type: nauc_recall_at_1_max
|
|
value: -13.351553937797
|
|
- type: nauc_recall_at_1_std
|
|
value: -14.137614923859612
|
|
- type: nauc_recall_at_20_diff1
|
|
value: -2.0846698569576856
|
|
- type: nauc_recall_at_20_max
|
|
value: 6.736841084303534
|
|
- type: nauc_recall_at_20_std
|
|
value: -0.3301527168878837
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 4.202256387521115
|
|
- type: nauc_recall_at_3_max
|
|
value: -14.043068948669694
|
|
- type: nauc_recall_at_3_std
|
|
value: -19.716257127342317
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 3.26941301005235
|
|
- type: nauc_recall_at_5_max
|
|
value: -12.777275311820102
|
|
- type: nauc_recall_at_5_std
|
|
value: -20.917228577779866
|
|
- type: ndcg_at_1
|
|
value: 31.080999999999996
|
|
- type: ndcg_at_10
|
|
value: 56.569
|
|
- type: ndcg_at_100
|
|
value: 59.772999999999996
|
|
- type: ndcg_at_1000
|
|
value: 59.843
|
|
- type: ndcg_at_20
|
|
value: 58.933
|
|
- type: ndcg_at_3
|
|
value: 46.209
|
|
- type: ndcg_at_5
|
|
value: 51.090999999999994
|
|
- type: precision_at_1
|
|
value: 31.080999999999996
|
|
- type: precision_at_10
|
|
value: 8.578
|
|
- type: precision_at_100
|
|
value: 0.991
|
|
- type: precision_at_1000
|
|
value: 0.1
|
|
- type: precision_at_20
|
|
value: 4.744000000000001
|
|
- type: precision_at_3
|
|
value: 19.061
|
|
- type: precision_at_5
|
|
value: 13.812
|
|
- type: recall_at_1
|
|
value: 31.080999999999996
|
|
- type: recall_at_10
|
|
value: 85.775
|
|
- type: recall_at_100
|
|
value: 99.14699999999999
|
|
- type: recall_at_1000
|
|
value: 99.644
|
|
- type: recall_at_20
|
|
value: 94.879
|
|
- type: recall_at_3
|
|
value: 57.18299999999999
|
|
- type: recall_at_5
|
|
value: 69.06099999999999
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB ArxivClusteringP2P
|
|
revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
|
|
split: test
|
|
type: mteb/arxiv-clustering-p2p
|
|
metrics:
|
|
- type: main_score
|
|
value: 48.009758343820856
|
|
- type: v_measure
|
|
value: 48.009758343820856
|
|
- type: v_measure_std
|
|
value: 14.203651443985635
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB ArxivClusteringS2S
|
|
revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
|
|
split: test
|
|
type: mteb/arxiv-clustering-s2s
|
|
metrics:
|
|
- type: main_score
|
|
value: 39.401811401341035
|
|
- type: v_measure
|
|
value: 39.401811401341035
|
|
- type: v_measure_std
|
|
value: 14.736655369522248
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB AskUbuntuDupQuestions
|
|
revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
|
|
split: test
|
|
type: mteb/askubuntudupquestions-reranking
|
|
metrics:
|
|
- type: main_score
|
|
value: 60.158996366210474
|
|
- type: map
|
|
value: 60.158996366210474
|
|
- type: mrr
|
|
value: 74.69034428175702
|
|
- type: nAUC_map_diff1
|
|
value: 7.7660414737755605
|
|
- type: nAUC_map_max
|
|
value: 20.377348037855818
|
|
- type: nAUC_map_std
|
|
value: 18.290516035806565
|
|
- type: nAUC_mrr_diff1
|
|
value: 10.721266751736124
|
|
- type: nAUC_mrr_max
|
|
value: 31.3686330442438
|
|
- type: nAUC_mrr_std
|
|
value: 19.240868443170196
|
|
task:
|
|
type: Reranking
|
|
- dataset:
|
|
config: default
|
|
name: MTEB BIOSSES
|
|
revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
|
|
split: test
|
|
type: mteb/biosses-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 87.53887478826596
|
|
- type: cosine_spearman
|
|
value: 86.32606338345799
|
|
- type: euclidean_pearson
|
|
value: 86.76233071291158
|
|
- type: euclidean_spearman
|
|
value: 86.32606338345799
|
|
- type: main_score
|
|
value: 86.32606338345799
|
|
- type: manhattan_pearson
|
|
value: 86.05455915524152
|
|
- type: manhattan_spearman
|
|
value: 85.8868967502423
|
|
- type: pearson
|
|
value: 87.53887478826596
|
|
- type: spearman
|
|
value: 86.32606338345799
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB Banking77Classification
|
|
revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
|
|
split: test
|
|
type: mteb/banking77
|
|
metrics:
|
|
- type: accuracy
|
|
value: 84.92857142857144
|
|
- type: f1
|
|
value: 84.30505630131526
|
|
- type: f1_weighted
|
|
value: 84.30505630131528
|
|
- type: main_score
|
|
value: 84.92857142857144
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB BiorxivClusteringP2P
|
|
revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
|
|
split: test
|
|
type: mteb/biorxiv-clustering-p2p
|
|
metrics:
|
|
- type: main_score
|
|
value: 40.014867273983484
|
|
- type: v_measure
|
|
value: 40.014867273983484
|
|
- type: v_measure_std
|
|
value: 0.6558905123714063
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB BiorxivClusteringS2S
|
|
revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
|
|
split: test
|
|
type: mteb/biorxiv-clustering-s2s
|
|
metrics:
|
|
- type: main_score
|
|
value: 33.79424302438114
|
|
- type: v_measure
|
|
value: 33.79424302438114
|
|
- type: v_measure_std
|
|
value: 0.837779778459544
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackAndroidRetrieval
|
|
revision: f46a197baaae43b4f621051089b82a364682dfeb
|
|
split: test
|
|
type: mteb/cqadupstack-android
|
|
metrics:
|
|
- type: main_score
|
|
value: 52.884
|
|
- type: map_at_1
|
|
value: 34.634
|
|
- type: map_at_10
|
|
value: 46.339000000000006
|
|
- type: map_at_100
|
|
value: 47.857
|
|
- type: map_at_1000
|
|
value: 47.97
|
|
- type: map_at_20
|
|
value: 47.205000000000005
|
|
- type: map_at_3
|
|
value: 42.543
|
|
- type: map_at_5
|
|
value: 44.772
|
|
- type: mrr_at_1
|
|
value: 41.3447782546495
|
|
- type: mrr_at_10
|
|
value: 52.23857210981671
|
|
- type: mrr_at_100
|
|
value: 52.90915062899396
|
|
- type: mrr_at_1000
|
|
value: 52.95240146583995
|
|
- type: mrr_at_20
|
|
value: 52.655345331835804
|
|
- type: mrr_at_3
|
|
value: 49.71387696709583
|
|
- type: mrr_at_5
|
|
value: 51.23748211731041
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 51.49705927936061
|
|
- type: nauc_map_at_1000_max
|
|
value: 35.528845247090466
|
|
- type: nauc_map_at_1000_std
|
|
value: -4.253741985593714
|
|
- type: nauc_map_at_100_diff1
|
|
value: 51.508584685268886
|
|
- type: nauc_map_at_100_max
|
|
value: 35.56248075672379
|
|
- type: nauc_map_at_100_std
|
|
value: -4.176500881199186
|
|
- type: nauc_map_at_10_diff1
|
|
value: 51.338718973920614
|
|
- type: nauc_map_at_10_max
|
|
value: 34.946543347441214
|
|
- type: nauc_map_at_10_std
|
|
value: -5.33037717427031
|
|
- type: nauc_map_at_1_diff1
|
|
value: 56.23620820617472
|
|
- type: nauc_map_at_1_max
|
|
value: 30.320970401424987
|
|
- type: nauc_map_at_1_std
|
|
value: -6.655365474007067
|
|
- type: nauc_map_at_20_diff1
|
|
value: 51.51775947048102
|
|
- type: nauc_map_at_20_max
|
|
value: 35.25983470448141
|
|
- type: nauc_map_at_20_std
|
|
value: -4.612859125963163
|
|
- type: nauc_map_at_3_diff1
|
|
value: 52.725269902770755
|
|
- type: nauc_map_at_3_max
|
|
value: 33.299803481018195
|
|
- type: nauc_map_at_3_std
|
|
value: -6.33353874546021
|
|
- type: nauc_map_at_5_diff1
|
|
value: 51.672349084315485
|
|
- type: nauc_map_at_5_max
|
|
value: 34.645370794379886
|
|
- type: nauc_map_at_5_std
|
|
value: -5.94117791112353
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 49.12249635354981
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 36.29480359532615
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -4.665759763477847
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 49.11255442003998
|
|
- type: nauc_mrr_at_100_max
|
|
value: 36.29806935257465
|
|
- type: nauc_mrr_at_100_std
|
|
value: -4.663481407381479
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 48.779673215220065
|
|
- type: nauc_mrr_at_10_max
|
|
value: 36.12766214960087
|
|
- type: nauc_mrr_at_10_std
|
|
value: -5.0778877090392625
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 53.70331290003521
|
|
- type: nauc_mrr_at_1_max
|
|
value: 35.17671705244682
|
|
- type: nauc_mrr_at_1_std
|
|
value: -6.289432335416569
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 48.98440189775321
|
|
- type: nauc_mrr_at_20_max
|
|
value: 36.24567442841102
|
|
- type: nauc_mrr_at_20_std
|
|
value: -4.808524080549843
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 50.09142180621504
|
|
- type: nauc_mrr_at_3_max
|
|
value: 36.57201237478509
|
|
- type: nauc_mrr_at_3_std
|
|
value: -5.10258589719658
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 49.15413181233011
|
|
- type: nauc_mrr_at_5_max
|
|
value: 36.64387975128488
|
|
- type: nauc_mrr_at_5_std
|
|
value: -5.2142664019104
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 49.48338541267995
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 36.71225124867686
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -2.1565353674328636
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 49.378803009143354
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 37.05072158645242
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -1.1554881315239078
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 48.217255194293706
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 35.70709987917217
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -4.5843409864100835
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 53.70331290003521
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 35.17671705244682
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -6.289432335416569
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 48.90479671421663
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 35.63061062961699
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -3.2759049624453924
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 50.66992100707998
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 35.647144096807054
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -4.675684277632912
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 48.86023024957704
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 36.36204191994049
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -4.979721506683613
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -20.176146428291695
|
|
- type: nauc_precision_at_1000_max
|
|
value: -4.944333530911747
|
|
- type: nauc_precision_at_1000_std
|
|
value: -2.6416464331580256
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -11.455305661135391
|
|
- type: nauc_precision_at_100_max
|
|
value: 9.563783942313348
|
|
- type: nauc_precision_at_100_std
|
|
value: 9.987888995757324
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 6.577302086017673
|
|
- type: nauc_precision_at_10_max
|
|
value: 25.67586949524924
|
|
- type: nauc_precision_at_10_std
|
|
value: 5.543682394632135
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 53.70331290003521
|
|
- type: nauc_precision_at_1_max
|
|
value: 35.17671705244682
|
|
- type: nauc_precision_at_1_std
|
|
value: -6.289432335416569
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 0.0352451246393809
|
|
- type: nauc_precision_at_20_max
|
|
value: 19.02340589034973
|
|
- type: nauc_precision_at_20_std
|
|
value: 10.156322995661567
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 31.114868446262108
|
|
- type: nauc_precision_at_3_max
|
|
value: 35.740653736733925
|
|
- type: nauc_precision_at_3_std
|
|
value: -0.4754489918596968
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 17.05966182310583
|
|
- type: nauc_precision_at_5_max
|
|
value: 32.37346687203089
|
|
- type: nauc_precision_at_5_std
|
|
value: 1.4954175443689899
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 42.86116448480766
|
|
- type: nauc_recall_at_1000_max
|
|
value: 63.759509563968976
|
|
- type: nauc_recall_at_1000_std
|
|
value: 61.175429354991614
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 40.88375670987642
|
|
- type: nauc_recall_at_100_max
|
|
value: 44.62608189829668
|
|
- type: nauc_recall_at_100_std
|
|
value: 25.55163256804942
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 37.759771219935175
|
|
- type: nauc_recall_at_10_max
|
|
value: 31.146081092167627
|
|
- type: nauc_recall_at_10_std
|
|
value: -4.512890345394815
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 56.23620820617472
|
|
- type: nauc_recall_at_1_max
|
|
value: 30.320970401424987
|
|
- type: nauc_recall_at_1_std
|
|
value: -6.655365474007067
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 38.4827047216752
|
|
- type: nauc_recall_at_20_max
|
|
value: 30.50125803520275
|
|
- type: nauc_recall_at_20_std
|
|
value: 0.8771358044937425
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 47.487079446530906
|
|
- type: nauc_recall_at_3_max
|
|
value: 32.19896007873808
|
|
- type: nauc_recall_at_3_std
|
|
value: -5.164803420738882
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 41.699415045286415
|
|
- type: nauc_recall_at_5_max
|
|
value: 33.168829040464196
|
|
- type: nauc_recall_at_5_std
|
|
value: -5.366546702094067
|
|
- type: ndcg_at_1
|
|
value: 41.345
|
|
- type: ndcg_at_10
|
|
value: 52.884
|
|
- type: ndcg_at_100
|
|
value: 57.94200000000001
|
|
- type: ndcg_at_1000
|
|
value: 59.68
|
|
- type: ndcg_at_20
|
|
value: 54.957
|
|
- type: ndcg_at_3
|
|
value: 47.692
|
|
- type: ndcg_at_5
|
|
value: 50.251000000000005
|
|
- type: precision_at_1
|
|
value: 41.345
|
|
- type: precision_at_10
|
|
value: 10.057
|
|
- type: precision_at_100
|
|
value: 1.574
|
|
- type: precision_at_1000
|
|
value: 0.201
|
|
- type: precision_at_20
|
|
value: 5.9799999999999995
|
|
- type: precision_at_3
|
|
value: 22.842000000000002
|
|
- type: precision_at_5
|
|
value: 16.595
|
|
- type: recall_at_1
|
|
value: 34.634
|
|
- type: recall_at_10
|
|
value: 65.185
|
|
- type: recall_at_100
|
|
value: 85.703
|
|
- type: recall_at_1000
|
|
value: 96.65599999999999
|
|
- type: recall_at_20
|
|
value: 72.322
|
|
- type: recall_at_3
|
|
value: 50.182
|
|
- type: recall_at_5
|
|
value: 57.159000000000006
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackEnglishRetrieval
|
|
revision: ad9991cb51e31e31e430383c75ffb2885547b5f0
|
|
split: test
|
|
type: mteb/cqadupstack-english
|
|
metrics:
|
|
- type: main_score
|
|
value: 48.264
|
|
- type: map_at_1
|
|
value: 31.224
|
|
- type: map_at_10
|
|
value: 42.332
|
|
- type: map_at_100
|
|
value: 43.533
|
|
- type: map_at_1000
|
|
value: 43.662
|
|
- type: map_at_20
|
|
value: 42.972
|
|
- type: map_at_3
|
|
value: 39.159
|
|
- type: map_at_5
|
|
value: 41.047
|
|
- type: mrr_at_1
|
|
value: 39.04458598726115
|
|
- type: mrr_at_10
|
|
value: 48.18686179354971
|
|
- type: mrr_at_100
|
|
value: 48.803902946647234
|
|
- type: mrr_at_1000
|
|
value: 48.84702137486075
|
|
- type: mrr_at_20
|
|
value: 48.56368295512913
|
|
- type: mrr_at_3
|
|
value: 45.83864118895968
|
|
- type: mrr_at_5
|
|
value: 47.20806794055207
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 51.86414274615986
|
|
- type: nauc_map_at_1000_max
|
|
value: 34.717053941484025
|
|
- type: nauc_map_at_1000_std
|
|
value: -4.340680651811943
|
|
- type: nauc_map_at_100_diff1
|
|
value: 51.84970191815774
|
|
- type: nauc_map_at_100_max
|
|
value: 34.64676814212115
|
|
- type: nauc_map_at_100_std
|
|
value: -4.4387297635880385
|
|
- type: nauc_map_at_10_diff1
|
|
value: 52.119436277416945
|
|
- type: nauc_map_at_10_max
|
|
value: 33.94135075756255
|
|
- type: nauc_map_at_10_std
|
|
value: -5.625757602694689
|
|
- type: nauc_map_at_1_diff1
|
|
value: 57.92845763299044
|
|
- type: nauc_map_at_1_max
|
|
value: 29.3199164115535
|
|
- type: nauc_map_at_1_std
|
|
value: -11.586283921183611
|
|
- type: nauc_map_at_20_diff1
|
|
value: 51.92734614822424
|
|
- type: nauc_map_at_20_max
|
|
value: 34.35250084161699
|
|
- type: nauc_map_at_20_std
|
|
value: -5.049283716884917
|
|
- type: nauc_map_at_3_diff1
|
|
value: 52.783776450874356
|
|
- type: nauc_map_at_3_max
|
|
value: 32.394255917655535
|
|
- type: nauc_map_at_3_std
|
|
value: -8.05902730660978
|
|
- type: nauc_map_at_5_diff1
|
|
value: 52.14993873615333
|
|
- type: nauc_map_at_5_max
|
|
value: 33.48431923578608
|
|
- type: nauc_map_at_5_std
|
|
value: -6.472903440360678
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 50.49829531271091
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 37.183131918098425
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 0.4928095353543418
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 50.494636141021424
|
|
- type: nauc_mrr_at_100_max
|
|
value: 37.185446950719715
|
|
- type: nauc_mrr_at_100_std
|
|
value: 0.5056844413835279
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 50.55418166759066
|
|
- type: nauc_mrr_at_10_max
|
|
value: 37.17369235180479
|
|
- type: nauc_mrr_at_10_std
|
|
value: 0.3511264489316608
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 55.09381247060509
|
|
- type: nauc_mrr_at_1_max
|
|
value: 37.17089033507927
|
|
- type: nauc_mrr_at_1_std
|
|
value: -2.545073558300969
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 50.46232349188045
|
|
- type: nauc_mrr_at_20_max
|
|
value: 37.22028938157565
|
|
- type: nauc_mrr_at_20_std
|
|
value: 0.4342508184428254
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 50.98797216868357
|
|
- type: nauc_mrr_at_3_max
|
|
value: 37.32821622965925
|
|
- type: nauc_mrr_at_3_std
|
|
value: -0.6918122573096884
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 50.477903924122025
|
|
- type: nauc_mrr_at_5_max
|
|
value: 37.343161615517296
|
|
- type: nauc_mrr_at_5_std
|
|
value: 0.34187371397979793
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 49.71083273417971
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 36.08714449707927
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 0.3359295264579242
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 49.64047591726873
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 36.0502827680962
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 0.4394585830222923
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 50.3895028633975
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 35.51838515595454
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -1.8340842845181509
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 55.09381247060509
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 37.17089033507927
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -2.545073558300969
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 49.975850062007375
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 35.8777155711073
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -1.1833564484981665
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 50.3823214340417
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 35.776477162991746
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -3.0969092422279623
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 50.18424405483706
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 35.886540678742485
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -2.2048728336054912
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -8.409825453277659
|
|
- type: nauc_precision_at_1000_max
|
|
value: 14.148796859940632
|
|
- type: nauc_precision_at_1000_std
|
|
value: 28.34712816378856
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -4.133099395945424
|
|
- type: nauc_precision_at_100_max
|
|
value: 23.436894225838895
|
|
- type: nauc_precision_at_100_std
|
|
value: 31.777687917658554
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 12.456499608847746
|
|
- type: nauc_precision_at_10_max
|
|
value: 34.40385767678226
|
|
- type: nauc_precision_at_10_std
|
|
value: 22.64168731207244
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 55.09381247060509
|
|
- type: nauc_precision_at_1_max
|
|
value: 37.17089033507927
|
|
- type: nauc_precision_at_1_std
|
|
value: -2.545073558300969
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 4.838516065171166
|
|
- type: nauc_precision_at_20_max
|
|
value: 31.381417947568412
|
|
- type: nauc_precision_at_20_std
|
|
value: 26.974660907322917
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 28.180760599976384
|
|
- type: nauc_precision_at_3_max
|
|
value: 36.40321247194992
|
|
- type: nauc_precision_at_3_std
|
|
value: 9.375871028699667
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 19.689988735115058
|
|
- type: nauc_precision_at_5_max
|
|
value: 35.98837508752083
|
|
- type: nauc_precision_at_5_std
|
|
value: 16.284464606894232
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 33.594125915695884
|
|
- type: nauc_recall_at_1000_max
|
|
value: 31.574941156196807
|
|
- type: nauc_recall_at_1000_std
|
|
value: 20.460707032380316
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 38.54327301097089
|
|
- type: nauc_recall_at_100_max
|
|
value: 33.368528599783126
|
|
- type: nauc_recall_at_100_std
|
|
value: 15.321500393966641
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 44.219731053687255
|
|
- type: nauc_recall_at_10_max
|
|
value: 31.484342080988824
|
|
- type: nauc_recall_at_10_std
|
|
value: 0.22452148883121484
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 57.92845763299044
|
|
- type: nauc_recall_at_1_max
|
|
value: 29.3199164115535
|
|
- type: nauc_recall_at_1_std
|
|
value: -11.586283921183611
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 41.39285600168573
|
|
- type: nauc_recall_at_20_max
|
|
value: 32.966202138611465
|
|
- type: nauc_recall_at_20_std
|
|
value: 3.365583403518244
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 47.33546382576856
|
|
- type: nauc_recall_at_3_max
|
|
value: 30.988541475501425
|
|
- type: nauc_recall_at_3_std
|
|
value: -5.87940259105687
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 45.27313627261692
|
|
- type: nauc_recall_at_5_max
|
|
value: 32.34545008582682
|
|
- type: nauc_recall_at_5_std
|
|
value: -1.6738776274622713
|
|
- type: ndcg_at_1
|
|
value: 39.045
|
|
- type: ndcg_at_10
|
|
value: 48.264
|
|
- type: ndcg_at_100
|
|
value: 52.493
|
|
- type: ndcg_at_1000
|
|
value: 54.457
|
|
- type: ndcg_at_20
|
|
value: 49.888
|
|
- type: ndcg_at_3
|
|
value: 43.86
|
|
- type: ndcg_at_5
|
|
value: 45.983000000000004
|
|
- type: precision_at_1
|
|
value: 39.045
|
|
- type: precision_at_10
|
|
value: 9.096
|
|
- type: precision_at_100
|
|
value: 1.442
|
|
- type: precision_at_1000
|
|
value: 0.191
|
|
- type: precision_at_20
|
|
value: 5.309
|
|
- type: precision_at_3
|
|
value: 21.316
|
|
- type: precision_at_5
|
|
value: 15.197
|
|
- type: recall_at_1
|
|
value: 31.224
|
|
- type: recall_at_10
|
|
value: 59.080999999999996
|
|
- type: recall_at_100
|
|
value: 76.897
|
|
- type: recall_at_1000
|
|
value: 89.23
|
|
- type: recall_at_20
|
|
value: 64.891
|
|
- type: recall_at_3
|
|
value: 46.076
|
|
- type: recall_at_5
|
|
value: 51.964
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackGamingRetrieval
|
|
revision: 4885aa143210c98657558c04aaf3dc47cfb54340
|
|
split: test
|
|
type: mteb/cqadupstack-gaming
|
|
metrics:
|
|
- type: main_score
|
|
value: 62.366
|
|
- type: map_at_1
|
|
value: 42.703
|
|
- type: map_at_10
|
|
value: 56.281000000000006
|
|
- type: map_at_100
|
|
value: 57.260999999999996
|
|
- type: map_at_1000
|
|
value: 57.30800000000001
|
|
- type: map_at_20
|
|
value: 56.871
|
|
- type: map_at_3
|
|
value: 52.897000000000006
|
|
- type: map_at_5
|
|
value: 54.773
|
|
- type: mrr_at_1
|
|
value: 48.589341692789965
|
|
- type: mrr_at_10
|
|
value: 59.43538836642291
|
|
- type: mrr_at_100
|
|
value: 59.999373625798235
|
|
- type: mrr_at_1000
|
|
value: 60.02341349127948
|
|
- type: mrr_at_20
|
|
value: 59.78236245014694
|
|
- type: mrr_at_3
|
|
value: 56.99059561128534
|
|
- type: mrr_at_5
|
|
value: 58.373040752351216
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 51.724911969542475
|
|
- type: nauc_map_at_1000_max
|
|
value: 31.59720256654406
|
|
- type: nauc_map_at_1000_std
|
|
value: -8.448863423330733
|
|
- type: nauc_map_at_100_diff1
|
|
value: 51.721207885585294
|
|
- type: nauc_map_at_100_max
|
|
value: 31.598189555174677
|
|
- type: nauc_map_at_100_std
|
|
value: -8.415293705149518
|
|
- type: nauc_map_at_10_diff1
|
|
value: 51.74316546903847
|
|
- type: nauc_map_at_10_max
|
|
value: 31.370796021816087
|
|
- type: nauc_map_at_10_std
|
|
value: -9.144187110965651
|
|
- type: nauc_map_at_1_diff1
|
|
value: 55.602123379999405
|
|
- type: nauc_map_at_1_max
|
|
value: 26.15423784568626
|
|
- type: nauc_map_at_1_std
|
|
value: -11.354042579102689
|
|
- type: nauc_map_at_20_diff1
|
|
value: 51.71343659271482
|
|
- type: nauc_map_at_20_max
|
|
value: 31.53988815092091
|
|
- type: nauc_map_at_20_std
|
|
value: -8.65212495986148
|
|
- type: nauc_map_at_3_diff1
|
|
value: 52.064639443577846
|
|
- type: nauc_map_at_3_max
|
|
value: 30.3485604522721
|
|
- type: nauc_map_at_3_std
|
|
value: -10.751274075635509
|
|
- type: nauc_map_at_5_diff1
|
|
value: 51.72321940513861
|
|
- type: nauc_map_at_5_max
|
|
value: 30.392319659455435
|
|
- type: nauc_map_at_5_std
|
|
value: -9.939778501885101
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 51.184984251728025
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 32.69216958808548
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -8.500776574802599
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 51.17941032241811
|
|
- type: nauc_mrr_at_100_max
|
|
value: 32.70608756736136
|
|
- type: nauc_mrr_at_100_std
|
|
value: -8.477679942920167
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 51.07904444322852
|
|
- type: nauc_mrr_at_10_max
|
|
value: 32.65962497893277
|
|
- type: nauc_mrr_at_10_std
|
|
value: -8.709383804816481
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 54.53142920528978
|
|
- type: nauc_mrr_at_1_max
|
|
value: 30.926785799334677
|
|
- type: nauc_mrr_at_1_std
|
|
value: -10.41145527848442
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 51.14693383001116
|
|
- type: nauc_mrr_at_20_max
|
|
value: 32.73093259139165
|
|
- type: nauc_mrr_at_20_std
|
|
value: -8.447633887171534
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 51.17432400675771
|
|
- type: nauc_mrr_at_3_max
|
|
value: 32.85252288214242
|
|
- type: nauc_mrr_at_3_std
|
|
value: -9.21642979066159
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 51.036935248981905
|
|
- type: nauc_mrr_at_5_max
|
|
value: 32.502626235077095
|
|
- type: nauc_mrr_at_5_std
|
|
value: -8.948887571702919
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 50.73024891705996
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 33.26584662078177
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -6.163854205845618
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 50.67040290788501
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 33.68165097437155
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -5.301942481514177
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 50.407269736351054
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 33.1723247102446
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -7.313191608002288
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 54.53142920528978
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 30.926785799334677
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -10.41145527848442
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 50.45722009686969
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 33.54250850995858
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -6.008420175252642
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 50.769657622259686
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 31.792120043553002
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -10.040327445335686
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 50.398976656987614
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 31.61780666125045
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -8.943124136769121
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -17.275717791952
|
|
- type: nauc_precision_at_1000_max
|
|
value: 7.275527027803384
|
|
- type: nauc_precision_at_1000_std
|
|
value: 16.685486896410826
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -11.162266422032406
|
|
- type: nauc_precision_at_100_max
|
|
value: 12.70258577369679
|
|
- type: nauc_precision_at_100_std
|
|
value: 21.391285680664513
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 7.81828602576801
|
|
- type: nauc_precision_at_10_max
|
|
value: 24.78598247621288
|
|
- type: nauc_precision_at_10_std
|
|
value: 9.374021745818432
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 54.53142920528978
|
|
- type: nauc_precision_at_1_max
|
|
value: 30.926785799334677
|
|
- type: nauc_precision_at_1_std
|
|
value: -10.41145527848442
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 0.1631191398252266
|
|
- type: nauc_precision_at_20_max
|
|
value: 20.619391150501272
|
|
- type: nauc_precision_at_20_std
|
|
value: 16.276264697116872
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 27.04714503298839
|
|
- type: nauc_precision_at_3_max
|
|
value: 30.101606964258337
|
|
- type: nauc_precision_at_3_std
|
|
value: -3.681729229946907
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 17.843974173274304
|
|
- type: nauc_precision_at_5_max
|
|
value: 25.676881643654763
|
|
- type: nauc_precision_at_5_std
|
|
value: 1.5965157990195873
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 29.087262485289735
|
|
- type: nauc_recall_at_1000_max
|
|
value: 59.55059060998873
|
|
- type: nauc_recall_at_1000_std
|
|
value: 62.21218125216127
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 41.30594954847261
|
|
- type: nauc_recall_at_100_max
|
|
value: 48.03865105456248
|
|
- type: nauc_recall_at_100_std
|
|
value: 28.904820877938946
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 43.528832373563795
|
|
- type: nauc_recall_at_10_max
|
|
value: 36.333747103215266
|
|
- type: nauc_recall_at_10_std
|
|
value: -0.586937217589867
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 55.602123379999405
|
|
- type: nauc_recall_at_1_max
|
|
value: 26.15423784568626
|
|
- type: nauc_recall_at_1_std
|
|
value: -11.354042579102689
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 42.86486871096986
|
|
- type: nauc_recall_at_20_max
|
|
value: 39.37052680687811
|
|
- type: nauc_recall_at_20_std
|
|
value: 7.7270172598031985
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 46.744057097749746
|
|
- type: nauc_recall_at_3_max
|
|
value: 32.0901543978326
|
|
- type: nauc_recall_at_3_std
|
|
value: -9.836059759091158
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 44.52443640046374
|
|
- type: nauc_recall_at_5_max
|
|
value: 31.155871822952808
|
|
- type: nauc_recall_at_5_std
|
|
value: -7.116612032547676
|
|
- type: ndcg_at_1
|
|
value: 48.589
|
|
- type: ndcg_at_10
|
|
value: 62.366
|
|
- type: ndcg_at_100
|
|
value: 66.011
|
|
- type: ndcg_at_1000
|
|
value: 66.88199999999999
|
|
- type: ndcg_at_20
|
|
value: 63.979
|
|
- type: ndcg_at_3
|
|
value: 56.764
|
|
- type: ndcg_at_5
|
|
value: 59.426
|
|
- type: precision_at_1
|
|
value: 48.589
|
|
- type: precision_at_10
|
|
value: 9.981
|
|
- type: precision_at_100
|
|
value: 1.277
|
|
- type: precision_at_1000
|
|
value: 0.13899999999999998
|
|
- type: precision_at_20
|
|
value: 5.514
|
|
- type: precision_at_3
|
|
value: 25.308000000000003
|
|
- type: precision_at_5
|
|
value: 17.241
|
|
- type: recall_at_1
|
|
value: 42.703
|
|
- type: recall_at_10
|
|
value: 77.08
|
|
- type: recall_at_100
|
|
value: 92.374
|
|
- type: recall_at_1000
|
|
value: 98.402
|
|
- type: recall_at_20
|
|
value: 82.87400000000001
|
|
- type: recall_at_3
|
|
value: 62.138000000000005
|
|
- type: recall_at_5
|
|
value: 68.679
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackGisRetrieval
|
|
revision: 5003b3064772da1887988e05400cf3806fe491f2
|
|
split: test
|
|
type: mteb/cqadupstack-gis
|
|
metrics:
|
|
- type: main_score
|
|
value: 39.971000000000004
|
|
- type: map_at_1
|
|
value: 25.06
|
|
- type: map_at_10
|
|
value: 34.551
|
|
- type: map_at_100
|
|
value: 35.568
|
|
- type: map_at_1000
|
|
value: 35.65
|
|
- type: map_at_20
|
|
value: 35.127
|
|
- type: map_at_3
|
|
value: 31.936999999999998
|
|
- type: map_at_5
|
|
value: 33.186
|
|
- type: mrr_at_1
|
|
value: 27.11864406779661
|
|
- type: mrr_at_10
|
|
value: 36.72652676889963
|
|
- type: mrr_at_100
|
|
value: 37.57204686098606
|
|
- type: mrr_at_1000
|
|
value: 37.63141267969674
|
|
- type: mrr_at_20
|
|
value: 37.19310670147632
|
|
- type: mrr_at_3
|
|
value: 34.27495291902072
|
|
- type: mrr_at_5
|
|
value: 35.438794726930304
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 43.63829107634628
|
|
- type: nauc_map_at_1000_max
|
|
value: 23.954060999822257
|
|
- type: nauc_map_at_1000_std
|
|
value: -0.5807446969781898
|
|
- type: nauc_map_at_100_diff1
|
|
value: 43.610748406014466
|
|
- type: nauc_map_at_100_max
|
|
value: 23.94949736158448
|
|
- type: nauc_map_at_100_std
|
|
value: -0.5982601848367343
|
|
- type: nauc_map_at_10_diff1
|
|
value: 43.72900243122612
|
|
- type: nauc_map_at_10_max
|
|
value: 23.508469522079885
|
|
- type: nauc_map_at_10_std
|
|
value: -0.5258931194184133
|
|
- type: nauc_map_at_1_diff1
|
|
value: 50.922871467903654
|
|
- type: nauc_map_at_1_max
|
|
value: 24.6067671408884
|
|
- type: nauc_map_at_1_std
|
|
value: -4.630126214452492
|
|
- type: nauc_map_at_20_diff1
|
|
value: 43.63024854824786
|
|
- type: nauc_map_at_20_max
|
|
value: 23.874524344212734
|
|
- type: nauc_map_at_20_std
|
|
value: -0.556366665388133
|
|
- type: nauc_map_at_3_diff1
|
|
value: 44.38253552931588
|
|
- type: nauc_map_at_3_max
|
|
value: 22.561513802056236
|
|
- type: nauc_map_at_3_std
|
|
value: -3.005119773408719
|
|
- type: nauc_map_at_5_diff1
|
|
value: 44.016586535650795
|
|
- type: nauc_map_at_5_max
|
|
value: 23.302456735449038
|
|
- type: nauc_map_at_5_std
|
|
value: -1.7618309245289323
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 42.68205493907015
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 26.024690905326025
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 0.6287706252427459
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 42.654961103491004
|
|
- type: nauc_mrr_at_100_max
|
|
value: 26.029087860328065
|
|
- type: nauc_mrr_at_100_std
|
|
value: 0.6163052064323858
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 42.56564515109072
|
|
- type: nauc_mrr_at_10_max
|
|
value: 25.666414824261224
|
|
- type: nauc_mrr_at_10_std
|
|
value: 0.7949641234835698
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 49.966125488185206
|
|
- type: nauc_mrr_at_1_max
|
|
value: 27.193710462071348
|
|
- type: nauc_mrr_at_1_std
|
|
value: -2.2786990240033718
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 42.65274684886744
|
|
- type: nauc_mrr_at_20_max
|
|
value: 26.052180768841172
|
|
- type: nauc_mrr_at_20_std
|
|
value: 0.7171447318848092
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 43.22408289408012
|
|
- type: nauc_mrr_at_3_max
|
|
value: 25.34061478734211
|
|
- type: nauc_mrr_at_3_std
|
|
value: -1.1093305128661515
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 42.87983482470224
|
|
- type: nauc_mrr_at_5_max
|
|
value: 25.91557396366082
|
|
- type: nauc_mrr_at_5_std
|
|
value: -0.13066697110897257
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 41.53426396594562
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 25.526814765685046
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 2.2841859589382487
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 40.61825803826763
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 25.344384823963455
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 1.9818508179504288
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 40.82184056229221
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 23.832384873845786
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 2.4835478280573966
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 49.966125488185206
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 27.193710462071348
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -2.2786990240033718
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 40.648257910495396
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 25.1143676738966
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 2.2994895733337084
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 42.115026070978224
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 22.895171049309084
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -2.160818780944711
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 41.608274106869516
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 23.8694881434902
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -0.2034244843217431
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -0.08291845059826138
|
|
- type: nauc_precision_at_1000_max
|
|
value: 20.313650012376964
|
|
- type: nauc_precision_at_1000_std
|
|
value: 13.510706405842074
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 9.885311318637227
|
|
- type: nauc_precision_at_100_max
|
|
value: 26.374081882816075
|
|
- type: nauc_precision_at_100_std
|
|
value: 12.021731392392521
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 25.883633917220507
|
|
- type: nauc_precision_at_10_max
|
|
value: 26.552638392568888
|
|
- type: nauc_precision_at_10_std
|
|
value: 14.460458912586468
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 49.966125488185206
|
|
- type: nauc_precision_at_1_max
|
|
value: 27.193710462071348
|
|
- type: nauc_precision_at_1_std
|
|
value: -2.2786990240033718
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 20.695053025711932
|
|
- type: nauc_precision_at_20_max
|
|
value: 29.151449538281586
|
|
- type: nauc_precision_at_20_std
|
|
value: 13.496486151691874
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 33.475423305252995
|
|
- type: nauc_precision_at_3_max
|
|
value: 24.486060318210537
|
|
- type: nauc_precision_at_3_std
|
|
value: 1.9847009660547001
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 31.14043721035368
|
|
- type: nauc_precision_at_5_max
|
|
value: 27.224889907879906
|
|
- type: nauc_precision_at_5_std
|
|
value: 6.539905565691817
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 34.33506268392135
|
|
- type: nauc_recall_at_1000_max
|
|
value: 37.11939420491589
|
|
- type: nauc_recall_at_1000_std
|
|
value: 31.371417780064085
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 26.348832193119886
|
|
- type: nauc_recall_at_100_max
|
|
value: 28.096364816659065
|
|
- type: nauc_recall_at_100_std
|
|
value: 11.980597075104523
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 31.684763745718985
|
|
- type: nauc_recall_at_10_max
|
|
value: 21.556273820201323
|
|
- type: nauc_recall_at_10_std
|
|
value: 10.480665669920347
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 50.922871467903654
|
|
- type: nauc_recall_at_1_max
|
|
value: 24.6067671408884
|
|
- type: nauc_recall_at_1_std
|
|
value: -4.630126214452492
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 30.160960913064304
|
|
- type: nauc_recall_at_20_max
|
|
value: 26.303437539000505
|
|
- type: nauc_recall_at_20_std
|
|
value: 10.389326804314718
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 36.88184391262179
|
|
- type: nauc_recall_at_3_max
|
|
value: 20.190953608016223
|
|
- type: nauc_recall_at_3_std
|
|
value: -1.3089868832214695
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 34.99254305849935
|
|
- type: nauc_recall_at_5_max
|
|
value: 22.230820355560727
|
|
- type: nauc_recall_at_5_std
|
|
value: 2.678023175693563
|
|
- type: ndcg_at_1
|
|
value: 27.119
|
|
- type: ndcg_at_10
|
|
value: 39.971000000000004
|
|
- type: ndcg_at_100
|
|
value: 44.952
|
|
- type: ndcg_at_1000
|
|
value: 46.821
|
|
- type: ndcg_at_20
|
|
value: 41.881
|
|
- type: ndcg_at_3
|
|
value: 34.727000000000004
|
|
- type: ndcg_at_5
|
|
value: 36.814
|
|
- type: precision_at_1
|
|
value: 27.119
|
|
- type: precision_at_10
|
|
value: 6.271
|
|
- type: precision_at_100
|
|
value: 0.9249999999999999
|
|
- type: precision_at_1000
|
|
value: 0.11199999999999999
|
|
- type: precision_at_20
|
|
value: 3.605
|
|
- type: precision_at_3
|
|
value: 14.991
|
|
- type: precision_at_5
|
|
value: 10.26
|
|
- type: recall_at_1
|
|
value: 25.06
|
|
- type: recall_at_10
|
|
value: 54.635
|
|
- type: recall_at_100
|
|
value: 77.639
|
|
- type: recall_at_1000
|
|
value: 91.301
|
|
- type: recall_at_20
|
|
value: 61.763
|
|
- type: recall_at_3
|
|
value: 40.143
|
|
- type: recall_at_5
|
|
value: 45.193
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackMathematicaRetrieval
|
|
revision: 90fceea13679c63fe563ded68f3b6f06e50061de
|
|
split: test
|
|
type: mteb/cqadupstack-mathematica
|
|
metrics:
|
|
- type: main_score
|
|
value: 30.308
|
|
- type: map_at_1
|
|
value: 16.154
|
|
- type: map_at_10
|
|
value: 24.743000000000002
|
|
- type: map_at_100
|
|
value: 26.069
|
|
- type: map_at_1000
|
|
value: 26.197
|
|
- type: map_at_20
|
|
value: 25.46
|
|
- type: map_at_3
|
|
value: 21.816
|
|
- type: map_at_5
|
|
value: 23.443
|
|
- type: mrr_at_1
|
|
value: 20.149253731343283
|
|
- type: mrr_at_10
|
|
value: 29.10847547974411
|
|
- type: mrr_at_100
|
|
value: 30.13595361660887
|
|
- type: mrr_at_1000
|
|
value: 30.211025784243766
|
|
- type: mrr_at_20
|
|
value: 29.706267830545784
|
|
- type: mrr_at_3
|
|
value: 26.451077943615264
|
|
- type: mrr_at_5
|
|
value: 27.868988391376444
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 31.47263576493308
|
|
- type: nauc_map_at_1000_max
|
|
value: 18.49384617286511
|
|
- type: nauc_map_at_1000_std
|
|
value: 0.5754985941500461
|
|
- type: nauc_map_at_100_diff1
|
|
value: 31.44160594144755
|
|
- type: nauc_map_at_100_max
|
|
value: 18.46607563648124
|
|
- type: nauc_map_at_100_std
|
|
value: 0.5879794819886102
|
|
- type: nauc_map_at_10_diff1
|
|
value: 31.71626861875994
|
|
- type: nauc_map_at_10_max
|
|
value: 18.662179744257916
|
|
- type: nauc_map_at_10_std
|
|
value: -0.013163124651967131
|
|
- type: nauc_map_at_1_diff1
|
|
value: 37.33971420967126
|
|
- type: nauc_map_at_1_max
|
|
value: 17.543923177907566
|
|
- type: nauc_map_at_1_std
|
|
value: -0.6312070176608349
|
|
- type: nauc_map_at_20_diff1
|
|
value: 31.443960381506987
|
|
- type: nauc_map_at_20_max
|
|
value: 18.39695256653282
|
|
- type: nauc_map_at_20_std
|
|
value: 0.24204111048796523
|
|
- type: nauc_map_at_3_diff1
|
|
value: 32.66647821102399
|
|
- type: nauc_map_at_3_max
|
|
value: 17.166769100670678
|
|
- type: nauc_map_at_3_std
|
|
value: 0.2511302116485242
|
|
- type: nauc_map_at_5_diff1
|
|
value: 31.814363889022516
|
|
- type: nauc_map_at_5_max
|
|
value: 17.450292361372707
|
|
- type: nauc_map_at_5_std
|
|
value: -0.45123652210324744
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 31.885514197021163
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 18.697001653609462
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -0.7075589181761113
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 31.859235999194958
|
|
- type: nauc_mrr_at_100_max
|
|
value: 18.685923862530778
|
|
- type: nauc_mrr_at_100_std
|
|
value: -0.7027394321332194
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 32.00819090481358
|
|
- type: nauc_mrr_at_10_max
|
|
value: 18.858552402155677
|
|
- type: nauc_mrr_at_10_std
|
|
value: -0.8729017160389365
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 36.55463496530352
|
|
- type: nauc_mrr_at_1_max
|
|
value: 17.893580417517832
|
|
- type: nauc_mrr_at_1_std
|
|
value: -2.7268036629932895
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 31.79086317678036
|
|
- type: nauc_mrr_at_20_max
|
|
value: 18.72847970596078
|
|
- type: nauc_mrr_at_20_std
|
|
value: -0.7526268512949703
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 32.24844813811655
|
|
- type: nauc_mrr_at_3_max
|
|
value: 17.810304497390504
|
|
- type: nauc_mrr_at_3_std
|
|
value: -1.3573591649881485
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 32.29719658849603
|
|
- type: nauc_mrr_at_5_max
|
|
value: 18.01176246232617
|
|
- type: nauc_mrr_at_5_std
|
|
value: -1.3156140758149915
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 30.420235654700672
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 20.14284394608303
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 2.409633449702056
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 29.54867297316048
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 19.63470407851956
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 3.062730904774899
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 30.288655944213627
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 20.304033843092395
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 0.7042902099149692
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 36.55463496530352
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 17.893580417517832
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -2.7268036629932895
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 29.315712836253248
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 19.55539590463071
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 1.4238452417516618
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 31.54355638372054
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 17.766299875547816
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 0.28964137714040095
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 30.818060499932542
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 18.068091310151164
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -0.16020203299958868
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 1.8177927649439825
|
|
- type: nauc_precision_at_1000_max
|
|
value: 1.9156412467603505
|
|
- type: nauc_precision_at_1000_std
|
|
value: -1.0195378172264247
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 7.852064632368817
|
|
- type: nauc_precision_at_100_max
|
|
value: 11.41378732164787
|
|
- type: nauc_precision_at_100_std
|
|
value: 8.845589790612463
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 19.576158908850957
|
|
- type: nauc_precision_at_10_max
|
|
value: 22.963840017872794
|
|
- type: nauc_precision_at_10_std
|
|
value: 2.426835326713512
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 36.55463496530352
|
|
- type: nauc_precision_at_1_max
|
|
value: 17.893580417517832
|
|
- type: nauc_precision_at_1_std
|
|
value: -2.7268036629932895
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 15.305985286454149
|
|
- type: nauc_precision_at_20_max
|
|
value: 18.827005672571858
|
|
- type: nauc_precision_at_20_std
|
|
value: 3.992229421735929
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 26.358279542321966
|
|
- type: nauc_precision_at_3_max
|
|
value: 19.340749761958552
|
|
- type: nauc_precision_at_3_std
|
|
value: 0.8501109386129221
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 22.462129435924727
|
|
- type: nauc_precision_at_5_max
|
|
value: 18.890119720243188
|
|
- type: nauc_precision_at_5_std
|
|
value: 0.21756962337473482
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 25.079504569576184
|
|
- type: nauc_recall_at_1000_max
|
|
value: 36.71138367024086
|
|
- type: nauc_recall_at_1000_std
|
|
value: 18.882277140819067
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 19.980741195591563
|
|
- type: nauc_recall_at_100_max
|
|
value: 21.648381374802273
|
|
- type: nauc_recall_at_100_std
|
|
value: 14.541121099803092
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 24.61930855038573
|
|
- type: nauc_recall_at_10_max
|
|
value: 22.98083391642699
|
|
- type: nauc_recall_at_10_std
|
|
value: 2.860945348018573
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 37.33971420967126
|
|
- type: nauc_recall_at_1_max
|
|
value: 17.543923177907566
|
|
- type: nauc_recall_at_1_std
|
|
value: -0.6312070176608349
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 20.478434900407255
|
|
- type: nauc_recall_at_20_max
|
|
value: 20.439655780702832
|
|
- type: nauc_recall_at_20_std
|
|
value: 5.4039574030039885
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 27.845972047264578
|
|
- type: nauc_recall_at_3_max
|
|
value: 16.649682003649193
|
|
- type: nauc_recall_at_3_std
|
|
value: 2.171037068117454
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 26.76354795664187
|
|
- type: nauc_recall_at_5_max
|
|
value: 17.488511178851763
|
|
- type: nauc_recall_at_5_std
|
|
value: 0.7909085800561211
|
|
- type: ndcg_at_1
|
|
value: 20.149
|
|
- type: ndcg_at_10
|
|
value: 30.308
|
|
- type: ndcg_at_100
|
|
value: 36.361
|
|
- type: ndcg_at_1000
|
|
value: 39.128
|
|
- type: ndcg_at_20
|
|
value: 32.719
|
|
- type: ndcg_at_3
|
|
value: 24.969
|
|
- type: ndcg_at_5
|
|
value: 27.409
|
|
- type: precision_at_1
|
|
value: 20.149
|
|
- type: precision_at_10
|
|
value: 5.784000000000001
|
|
- type: precision_at_100
|
|
value: 1.011
|
|
- type: precision_at_1000
|
|
value: 0.13799999999999998
|
|
- type: precision_at_20
|
|
value: 3.5319999999999996
|
|
- type: precision_at_3
|
|
value: 12.106
|
|
- type: precision_at_5
|
|
value: 9.030000000000001
|
|
- type: recall_at_1
|
|
value: 16.154
|
|
- type: recall_at_10
|
|
value: 43.092000000000006
|
|
- type: recall_at_100
|
|
value: 68.998
|
|
- type: recall_at_1000
|
|
value: 88.127
|
|
- type: recall_at_20
|
|
value: 51.937999999999995
|
|
- type: recall_at_3
|
|
value: 28.473
|
|
- type: recall_at_5
|
|
value: 34.624
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackPhysicsRetrieval
|
|
revision: 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4
|
|
split: test
|
|
type: mteb/cqadupstack-physics
|
|
metrics:
|
|
- type: main_score
|
|
value: 46.931
|
|
- type: map_at_1
|
|
value: 30.036
|
|
- type: map_at_10
|
|
value: 40.753
|
|
- type: map_at_100
|
|
value: 42.098
|
|
- type: map_at_1000
|
|
value: 42.201
|
|
- type: map_at_20
|
|
value: 41.494
|
|
- type: map_at_3
|
|
value: 37.55
|
|
- type: map_at_5
|
|
value: 39.266
|
|
- type: mrr_at_1
|
|
value: 36.57362848893166
|
|
- type: mrr_at_10
|
|
value: 46.15953985058891
|
|
- type: mrr_at_100
|
|
value: 46.964409847048735
|
|
- type: mrr_at_1000
|
|
value: 47.006684152310186
|
|
- type: mrr_at_20
|
|
value: 46.63576095668375
|
|
- type: mrr_at_3
|
|
value: 43.39108116778952
|
|
- type: mrr_at_5
|
|
value: 45.0609560474815
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 50.008393482865934
|
|
- type: nauc_map_at_1000_max
|
|
value: 27.44292854668337
|
|
- type: nauc_map_at_1000_std
|
|
value: -1.1827744848485413
|
|
- type: nauc_map_at_100_diff1
|
|
value: 50.01593736030433
|
|
- type: nauc_map_at_100_max
|
|
value: 27.401227555060693
|
|
- type: nauc_map_at_100_std
|
|
value: -1.226830892874052
|
|
- type: nauc_map_at_10_diff1
|
|
value: 50.22186852707843
|
|
- type: nauc_map_at_10_max
|
|
value: 26.882005386152162
|
|
- type: nauc_map_at_10_std
|
|
value: -1.7817280491798217
|
|
- type: nauc_map_at_1_diff1
|
|
value: 53.70420852974904
|
|
- type: nauc_map_at_1_max
|
|
value: 25.134260139256465
|
|
- type: nauc_map_at_1_std
|
|
value: -5.16360510676616
|
|
- type: nauc_map_at_20_diff1
|
|
value: 50.03553131371993
|
|
- type: nauc_map_at_20_max
|
|
value: 27.028712351429306
|
|
- type: nauc_map_at_20_std
|
|
value: -1.4264982725018232
|
|
- type: nauc_map_at_3_diff1
|
|
value: 50.56170061459129
|
|
- type: nauc_map_at_3_max
|
|
value: 27.125222360081885
|
|
- type: nauc_map_at_3_std
|
|
value: -2.1772011676637457
|
|
- type: nauc_map_at_5_diff1
|
|
value: 50.55287654401218
|
|
- type: nauc_map_at_5_max
|
|
value: 27.179943148291034
|
|
- type: nauc_map_at_5_std
|
|
value: -1.9278191493666326
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 50.19001608358556
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 30.11015154646845
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -0.01731538046574592
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 50.17990723644671
|
|
- type: nauc_mrr_at_100_max
|
|
value: 30.08888004508371
|
|
- type: nauc_mrr_at_100_std
|
|
value: -0.03777479539357456
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 50.29875316793952
|
|
- type: nauc_mrr_at_10_max
|
|
value: 30.0700394599554
|
|
- type: nauc_mrr_at_10_std
|
|
value: -0.1129279328368799
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 53.13267349109123
|
|
- type: nauc_mrr_at_1_max
|
|
value: 29.600631965679142
|
|
- type: nauc_mrr_at_1_std
|
|
value: -1.0534342020289145
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 50.20426738346865
|
|
- type: nauc_mrr_at_20_max
|
|
value: 30.03033165917099
|
|
- type: nauc_mrr_at_20_std
|
|
value: -0.0990630706915973
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 50.44930547118647
|
|
- type: nauc_mrr_at_3_max
|
|
value: 30.18069271699821
|
|
- type: nauc_mrr_at_3_std
|
|
value: -0.4106548753200651
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 50.42405239937933
|
|
- type: nauc_mrr_at_5_max
|
|
value: 30.323511080797132
|
|
- type: nauc_mrr_at_5_std
|
|
value: -0.10914898852912731
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 48.4023648301636
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 29.372043713546457
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 1.4160068477128542
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 48.43331450594402
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 28.62936981969224
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 0.8983763064461433
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 49.03974183137114
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 27.134352966349006
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -0.7394110214476277
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 53.13267349109123
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 29.600631965679142
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -1.0534342020289145
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 48.48145045039039
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 27.312478220117836
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -0.08007639532022988
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 49.795198984753725
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 28.851373164423457
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -0.7114306314589505
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 49.76549299850904
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 28.333095297025384
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -0.6065340225903514
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -14.995860825405593
|
|
- type: nauc_precision_at_1000_max
|
|
value: 10.497503977177239
|
|
- type: nauc_precision_at_1000_std
|
|
value: 15.472908805216562
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -5.056728888436733
|
|
- type: nauc_precision_at_100_max
|
|
value: 16.225279572994932
|
|
- type: nauc_precision_at_100_std
|
|
value: 17.333024162674036
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 18.485355184593836
|
|
- type: nauc_precision_at_10_max
|
|
value: 21.53388484848657
|
|
- type: nauc_precision_at_10_std
|
|
value: 9.864926100512946
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 53.13267349109123
|
|
- type: nauc_precision_at_1_max
|
|
value: 29.600631965679142
|
|
- type: nauc_precision_at_1_std
|
|
value: -1.0534342020289145
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 9.420119338006966
|
|
- type: nauc_precision_at_20_max
|
|
value: 19.132214665647382
|
|
- type: nauc_precision_at_20_std
|
|
value: 13.168229893698857
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 34.51509644827664
|
|
- type: nauc_precision_at_3_max
|
|
value: 28.988501800675305
|
|
- type: nauc_precision_at_3_std
|
|
value: 6.887544108087535
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 29.728890633704864
|
|
- type: nauc_precision_at_5_max
|
|
value: 27.527807375891044
|
|
- type: nauc_precision_at_5_std
|
|
value: 8.615115789487243
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 21.30536250453658
|
|
- type: nauc_recall_at_1000_max
|
|
value: 45.66826079811565
|
|
- type: nauc_recall_at_1000_std
|
|
value: 40.948489257124734
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 36.41578755512283
|
|
- type: nauc_recall_at_100_max
|
|
value: 25.843843547872236
|
|
- type: nauc_recall_at_100_std
|
|
value: 9.98566808528975
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 42.73428373449279
|
|
- type: nauc_recall_at_10_max
|
|
value: 22.45723124505396
|
|
- type: nauc_recall_at_10_std
|
|
value: 0.6596133636511106
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 53.70420852974904
|
|
- type: nauc_recall_at_1_max
|
|
value: 25.134260139256465
|
|
- type: nauc_recall_at_1_std
|
|
value: -5.16360510676616
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 39.67103657607903
|
|
- type: nauc_recall_at_20_max
|
|
value: 21.767425036370714
|
|
- type: nauc_recall_at_20_std
|
|
value: 2.792759310018829
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 46.672591160111224
|
|
- type: nauc_recall_at_3_max
|
|
value: 26.876529270231792
|
|
- type: nauc_recall_at_3_std
|
|
value: -1.1160005181114536
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 45.77174277314153
|
|
- type: nauc_recall_at_5_max
|
|
value: 26.349199537996853
|
|
- type: nauc_recall_at_5_std
|
|
value: -0.09454430813945205
|
|
- type: ndcg_at_1
|
|
value: 36.574
|
|
- type: ndcg_at_10
|
|
value: 46.931
|
|
- type: ndcg_at_100
|
|
value: 52.40899999999999
|
|
- type: ndcg_at_1000
|
|
value: 54.31
|
|
- type: ndcg_at_20
|
|
value: 49.098000000000006
|
|
- type: ndcg_at_3
|
|
value: 41.585
|
|
- type: ndcg_at_5
|
|
value: 44.009
|
|
- type: precision_at_1
|
|
value: 36.574
|
|
- type: precision_at_10
|
|
value: 8.518
|
|
- type: precision_at_100
|
|
value: 1.317
|
|
- type: precision_at_1000
|
|
value: 0.167
|
|
- type: precision_at_20
|
|
value: 4.99
|
|
- type: precision_at_3
|
|
value: 19.794999999999998
|
|
- type: precision_at_5
|
|
value: 13.879
|
|
- type: recall_at_1
|
|
value: 30.036
|
|
- type: recall_at_10
|
|
value: 60.043
|
|
- type: recall_at_100
|
|
value: 82.78999999999999
|
|
- type: recall_at_1000
|
|
value: 95.017
|
|
- type: recall_at_20
|
|
value: 67.509
|
|
- type: recall_at_3
|
|
value: 44.769
|
|
- type: recall_at_5
|
|
value: 51.23
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackProgrammersRetrieval
|
|
revision: 6184bc1440d2dbc7612be22b50686b8826d22b32
|
|
split: test
|
|
type: mteb/cqadupstack-programmers
|
|
metrics:
|
|
- type: main_score
|
|
value: 43.147999999999996
|
|
- type: map_at_1
|
|
value: 27.299
|
|
- type: map_at_10
|
|
value: 37.441
|
|
- type: map_at_100
|
|
value: 38.977000000000004
|
|
- type: map_at_1000
|
|
value: 39.068999999999996
|
|
- type: map_at_20
|
|
value: 38.282
|
|
- type: map_at_3
|
|
value: 34.217
|
|
- type: map_at_5
|
|
value: 36.027
|
|
- type: mrr_at_1
|
|
value: 33.44748858447489
|
|
- type: mrr_at_10
|
|
value: 42.456738783793554
|
|
- type: mrr_at_100
|
|
value: 43.485313174917046
|
|
- type: mrr_at_1000
|
|
value: 43.52577210412886
|
|
- type: mrr_at_20
|
|
value: 43.02431629929082
|
|
- type: mrr_at_3
|
|
value: 39.72602739726027
|
|
- type: mrr_at_5
|
|
value: 41.32420091324198
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 42.430993099089214
|
|
- type: nauc_map_at_1000_max
|
|
value: 28.098034312926952
|
|
- type: nauc_map_at_1000_std
|
|
value: 3.231295090968473
|
|
- type: nauc_map_at_100_diff1
|
|
value: 42.42649976590143
|
|
- type: nauc_map_at_100_max
|
|
value: 28.07518501114065
|
|
- type: nauc_map_at_100_std
|
|
value: 3.2663627223954257
|
|
- type: nauc_map_at_10_diff1
|
|
value: 42.37108247761657
|
|
- type: nauc_map_at_10_max
|
|
value: 27.784006301694887
|
|
- type: nauc_map_at_10_std
|
|
value: 2.1562734801370382
|
|
- type: nauc_map_at_1_diff1
|
|
value: 46.996543750833226
|
|
- type: nauc_map_at_1_max
|
|
value: 23.22775877678291
|
|
- type: nauc_map_at_1_std
|
|
value: -3.185987618625673
|
|
- type: nauc_map_at_20_diff1
|
|
value: 42.285605547136605
|
|
- type: nauc_map_at_20_max
|
|
value: 27.87619604505037
|
|
- type: nauc_map_at_20_std
|
|
value: 2.868182127790041
|
|
- type: nauc_map_at_3_diff1
|
|
value: 43.17884748984982
|
|
- type: nauc_map_at_3_max
|
|
value: 26.640107029543174
|
|
- type: nauc_map_at_3_std
|
|
value: -0.6337177522670645
|
|
- type: nauc_map_at_5_diff1
|
|
value: 42.55295619170691
|
|
- type: nauc_map_at_5_max
|
|
value: 27.09386543850697
|
|
- type: nauc_map_at_5_std
|
|
value: 1.1172301120800785
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 41.44240071604904
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 29.942727017459177
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 4.847580130462551
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 41.43634208329461
|
|
- type: nauc_mrr_at_100_max
|
|
value: 29.94502158371524
|
|
- type: nauc_mrr_at_100_std
|
|
value: 4.873085525046516
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 41.434406767394215
|
|
- type: nauc_mrr_at_10_max
|
|
value: 29.961051443508534
|
|
- type: nauc_mrr_at_10_std
|
|
value: 4.490183376727645
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 46.01681006012476
|
|
- type: nauc_mrr_at_1_max
|
|
value: 28.39735171499139
|
|
- type: nauc_mrr_at_1_std
|
|
value: 0.8500045602957598
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 41.324947979964605
|
|
- type: nauc_mrr_at_20_max
|
|
value: 29.939799023317963
|
|
- type: nauc_mrr_at_20_std
|
|
value: 4.8458435024129685
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 41.87918200877444
|
|
- type: nauc_mrr_at_3_max
|
|
value: 29.878707844397507
|
|
- type: nauc_mrr_at_3_std
|
|
value: 2.754394941481161
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 41.17158211294708
|
|
- type: nauc_mrr_at_5_max
|
|
value: 29.525114418603625
|
|
- type: nauc_mrr_at_5_std
|
|
value: 3.6695976231626792
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 40.85015584223998
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 30.175833847400003
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 7.454581754774201
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 40.679563549502475
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 30.105638179098303
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 8.61962835140906
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 40.37700967457906
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 29.33300077317775
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 5.023758212980035
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 46.01681006012476
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 28.39735171499139
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 0.8500045602957598
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 39.98886010789604
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 29.36296219371212
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 7.1201782062536925
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 40.92324084648135
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 28.520942397787785
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 1.0293165278727892
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 40.317533959797814
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 28.339428764903264
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 3.1896497530161687
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -6.9969817860247625
|
|
- type: nauc_precision_at_1000_max
|
|
value: 9.347778794059506
|
|
- type: nauc_precision_at_1000_std
|
|
value: 7.9646208472184625
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 2.991937395454712
|
|
- type: nauc_precision_at_100_max
|
|
value: 18.71624281667294
|
|
- type: nauc_precision_at_100_std
|
|
value: 21.600526590609512
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 18.37445514123775
|
|
- type: nauc_precision_at_10_max
|
|
value: 29.699257376065063
|
|
- type: nauc_precision_at_10_std
|
|
value: 18.095751349204832
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 46.01681006012476
|
|
- type: nauc_precision_at_1_max
|
|
value: 28.39735171499139
|
|
- type: nauc_precision_at_1_std
|
|
value: 0.8500045602957598
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 11.472713745988054
|
|
- type: nauc_precision_at_20_max
|
|
value: 25.690985880662325
|
|
- type: nauc_precision_at_20_std
|
|
value: 22.46754877988948
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 29.052028827439607
|
|
- type: nauc_precision_at_3_max
|
|
value: 31.04481903220871
|
|
- type: nauc_precision_at_3_std
|
|
value: 8.208096616199493
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 23.711708272374533
|
|
- type: nauc_precision_at_5_max
|
|
value: 30.24946804680551
|
|
- type: nauc_precision_at_5_std
|
|
value: 12.681259000978528
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 16.82259171106293
|
|
- type: nauc_recall_at_1000_max
|
|
value: 42.76820203485854
|
|
- type: nauc_recall_at_1000_std
|
|
value: 55.97238149176407
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 27.21094062723115
|
|
- type: nauc_recall_at_100_max
|
|
value: 33.698956290459584
|
|
- type: nauc_recall_at_100_std
|
|
value: 37.63664733891902
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 33.26348363515544
|
|
- type: nauc_recall_at_10_max
|
|
value: 29.5718227632449
|
|
- type: nauc_recall_at_10_std
|
|
value: 10.62584355073482
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 46.996543750833226
|
|
- type: nauc_recall_at_1_max
|
|
value: 23.22775877678291
|
|
- type: nauc_recall_at_1_std
|
|
value: -3.185987618625673
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 30.615386537256107
|
|
- type: nauc_recall_at_20_max
|
|
value: 29.459243404458636
|
|
- type: nauc_recall_at_20_std
|
|
value: 18.849849153868913
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 37.22492629427872
|
|
- type: nauc_recall_at_3_max
|
|
value: 27.49351222866847
|
|
- type: nauc_recall_at_3_std
|
|
value: 0.31700586087567145
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 34.4555753891359
|
|
- type: nauc_recall_at_5_max
|
|
value: 27.219221048995283
|
|
- type: nauc_recall_at_5_std
|
|
value: 6.057763073329902
|
|
- type: ndcg_at_1
|
|
value: 33.446999999999996
|
|
- type: ndcg_at_10
|
|
value: 43.147999999999996
|
|
- type: ndcg_at_100
|
|
value: 49.601
|
|
- type: ndcg_at_1000
|
|
value: 51.437
|
|
- type: ndcg_at_20
|
|
value: 45.704
|
|
- type: ndcg_at_3
|
|
value: 37.978
|
|
- type: ndcg_at_5
|
|
value: 40.431
|
|
- type: precision_at_1
|
|
value: 33.446999999999996
|
|
- type: precision_at_10
|
|
value: 7.888000000000001
|
|
- type: precision_at_100
|
|
value: 1.298
|
|
- type: precision_at_1000
|
|
value: 0.16199999999999998
|
|
- type: precision_at_20
|
|
value: 4.749
|
|
- type: precision_at_3
|
|
value: 17.922
|
|
- type: precision_at_5
|
|
value: 12.9
|
|
- type: recall_at_1
|
|
value: 27.299
|
|
- type: recall_at_10
|
|
value: 54.92399999999999
|
|
- type: recall_at_100
|
|
value: 82.308
|
|
- type: recall_at_1000
|
|
value: 94.451
|
|
- type: recall_at_20
|
|
value: 63.952
|
|
- type: recall_at_3
|
|
value: 40.788000000000004
|
|
- type: recall_at_5
|
|
value: 47.198
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackRetrieval
|
|
revision: CQADupstackRetrieval_is_a_combined_dataset
|
|
split: test
|
|
type: CQADupstackRetrieval_is_a_combined_dataset
|
|
metrics:
|
|
- type: main_score
|
|
value: 42.21466666666666
|
|
- type: ndcg_at_10
|
|
value: 42.21466666666666
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackStatsRetrieval
|
|
revision: 65ac3a16b8e91f9cee4c9828cc7c335575432a2a
|
|
split: test
|
|
type: mteb/cqadupstack-stats
|
|
metrics:
|
|
- type: main_score
|
|
value: 35.535
|
|
- type: map_at_1
|
|
value: 23.082
|
|
- type: map_at_10
|
|
value: 30.991000000000003
|
|
- type: map_at_100
|
|
value: 31.968000000000004
|
|
- type: map_at_1000
|
|
value: 32.07
|
|
- type: map_at_20
|
|
value: 31.535000000000004
|
|
- type: map_at_3
|
|
value: 28.605000000000004
|
|
- type: map_at_5
|
|
value: 30.06
|
|
- type: mrr_at_1
|
|
value: 25.920245398773005
|
|
- type: mrr_at_10
|
|
value: 33.93191888207225
|
|
- type: mrr_at_100
|
|
value: 34.77251067424867
|
|
- type: mrr_at_1000
|
|
value: 34.838890717603476
|
|
- type: mrr_at_20
|
|
value: 34.396659782410225
|
|
- type: mrr_at_3
|
|
value: 31.722903885480587
|
|
- type: mrr_at_5
|
|
value: 33.11860940695298
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 50.959235687536655
|
|
- type: nauc_map_at_1000_max
|
|
value: 30.655083426929526
|
|
- type: nauc_map_at_1000_std
|
|
value: 4.551329335263164
|
|
- type: nauc_map_at_100_diff1
|
|
value: 50.95439619487166
|
|
- type: nauc_map_at_100_max
|
|
value: 30.623042271335667
|
|
- type: nauc_map_at_100_std
|
|
value: 4.553201745695824
|
|
- type: nauc_map_at_10_diff1
|
|
value: 50.67983398647876
|
|
- type: nauc_map_at_10_max
|
|
value: 30.286986966981583
|
|
- type: nauc_map_at_10_std
|
|
value: 3.9148983660544125
|
|
- type: nauc_map_at_1_diff1
|
|
value: 58.20205266764334
|
|
- type: nauc_map_at_1_max
|
|
value: 28.58134257489169
|
|
- type: nauc_map_at_1_std
|
|
value: 0.40198884745343
|
|
- type: nauc_map_at_20_diff1
|
|
value: 50.90472178620438
|
|
- type: nauc_map_at_20_max
|
|
value: 30.563325966498205
|
|
- type: nauc_map_at_20_std
|
|
value: 4.369655492671673
|
|
- type: nauc_map_at_3_diff1
|
|
value: 52.084512866747325
|
|
- type: nauc_map_at_3_max
|
|
value: 29.374244156637356
|
|
- type: nauc_map_at_3_std
|
|
value: 2.0818606642419963
|
|
- type: nauc_map_at_5_diff1
|
|
value: 51.27705609284862
|
|
- type: nauc_map_at_5_max
|
|
value: 30.17700495077409
|
|
- type: nauc_map_at_5_std
|
|
value: 3.2722185125269103
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 51.909591752092425
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 33.36453135370183
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 7.404496516950065
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 51.900856693619126
|
|
- type: nauc_mrr_at_100_max
|
|
value: 33.350334085938364
|
|
- type: nauc_mrr_at_100_std
|
|
value: 7.410015907741515
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 51.82074175684569
|
|
- type: nauc_mrr_at_10_max
|
|
value: 33.32820656085001
|
|
- type: nauc_mrr_at_10_std
|
|
value: 7.043558257826565
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 60.46456002140532
|
|
- type: nauc_mrr_at_1_max
|
|
value: 33.31049028455304
|
|
- type: nauc_mrr_at_1_std
|
|
value: 4.830131026566884
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 51.8644842944308
|
|
- type: nauc_mrr_at_20_max
|
|
value: 33.3675144190388
|
|
- type: nauc_mrr_at_20_std
|
|
value: 7.256444002173675
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 52.904828169011154
|
|
- type: nauc_mrr_at_3_max
|
|
value: 32.55024244450511
|
|
- type: nauc_mrr_at_3_std
|
|
value: 6.014060915782276
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 52.361187623943614
|
|
- type: nauc_mrr_at_5_max
|
|
value: 33.38079408144374
|
|
- type: nauc_mrr_at_5_std
|
|
value: 6.854165091950606
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 48.30949790825087
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 32.568281800544476
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 8.966636096573168
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 47.9550901718591
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 31.969231434862483
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 8.909343996509326
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 47.56929495928323
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 31.131109409439638
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 6.03049937873584
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 60.46456002140532
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 33.31049028455304
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 4.830131026566884
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 47.99938648902949
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 31.584023047520475
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 7.3552147944361685
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 50.28269131499986
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 30.233582570806007
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 3.78476869218036
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 49.049921852112895
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 31.174764383636816
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 4.931908749150788
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 6.883972818358869
|
|
- type: nauc_precision_at_1000_max
|
|
value: 21.834322765687677
|
|
- type: nauc_precision_at_1000_std
|
|
value: 20.000731976327703
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 19.786688523669632
|
|
- type: nauc_precision_at_100_max
|
|
value: 30.328428959273722
|
|
- type: nauc_precision_at_100_std
|
|
value: 26.147922491368902
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 31.41218497795092
|
|
- type: nauc_precision_at_10_max
|
|
value: 33.95003889463453
|
|
- type: nauc_precision_at_10_std
|
|
value: 19.08301072890509
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 60.46456002140532
|
|
- type: nauc_precision_at_1_max
|
|
value: 33.31049028455304
|
|
- type: nauc_precision_at_1_std
|
|
value: 4.830131026566884
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 30.502712564255486
|
|
- type: nauc_precision_at_20_max
|
|
value: 35.178872501427975
|
|
- type: nauc_precision_at_20_std
|
|
value: 23.358935743161783
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 43.1022211297112
|
|
- type: nauc_precision_at_3_max
|
|
value: 33.93732742672912
|
|
- type: nauc_precision_at_3_std
|
|
value: 10.823942310140167
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 38.63486834833309
|
|
- type: nauc_precision_at_5_max
|
|
value: 36.23894828623807
|
|
- type: nauc_precision_at_5_std
|
|
value: 14.675475211699615
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 23.04089688983766
|
|
- type: nauc_recall_at_1000_max
|
|
value: 40.167606539321355
|
|
- type: nauc_recall_at_1000_std
|
|
value: 43.02153663005034
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 32.000202612409794
|
|
- type: nauc_recall_at_100_max
|
|
value: 31.12741249551696
|
|
- type: nauc_recall_at_100_std
|
|
value: 24.54365478830203
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 36.14374447048929
|
|
- type: nauc_recall_at_10_max
|
|
value: 29.498316079260555
|
|
- type: nauc_recall_at_10_std
|
|
value: 8.641435315254533
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 58.20205266764334
|
|
- type: nauc_recall_at_1_max
|
|
value: 28.58134257489169
|
|
- type: nauc_recall_at_1_std
|
|
value: 0.40198884745343
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 36.22347557385489
|
|
- type: nauc_recall_at_20_max
|
|
value: 29.750817583764405
|
|
- type: nauc_recall_at_20_std
|
|
value: 13.219998916877149
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 43.42606106046774
|
|
- type: nauc_recall_at_3_max
|
|
value: 27.02370831585066
|
|
- type: nauc_recall_at_3_std
|
|
value: 2.148594878901326
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 40.74252027743906
|
|
- type: nauc_recall_at_5_max
|
|
value: 29.661893704694375
|
|
- type: nauc_recall_at_5_std
|
|
value: 5.8950594952457145
|
|
- type: ndcg_at_1
|
|
value: 25.919999999999998
|
|
- type: ndcg_at_10
|
|
value: 35.535
|
|
- type: ndcg_at_100
|
|
value: 40.316
|
|
- type: ndcg_at_1000
|
|
value: 42.84
|
|
- type: ndcg_at_20
|
|
value: 37.424
|
|
- type: ndcg_at_3
|
|
value: 31.223
|
|
- type: ndcg_at_5
|
|
value: 33.521
|
|
- type: precision_at_1
|
|
value: 25.919999999999998
|
|
- type: precision_at_10
|
|
value: 5.736
|
|
- type: precision_at_100
|
|
value: 0.876
|
|
- type: precision_at_1000
|
|
value: 0.117
|
|
- type: precision_at_20
|
|
value: 3.359
|
|
- type: precision_at_3
|
|
value: 13.804
|
|
- type: precision_at_5
|
|
value: 9.754999999999999
|
|
- type: recall_at_1
|
|
value: 23.082
|
|
- type: recall_at_10
|
|
value: 46.399
|
|
- type: recall_at_100
|
|
value: 68.06
|
|
- type: recall_at_1000
|
|
value: 86.821
|
|
- type: recall_at_20
|
|
value: 53.525
|
|
- type: recall_at_3
|
|
value: 34.871
|
|
- type: recall_at_5
|
|
value: 40.492
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackTexRetrieval
|
|
revision: 46989137a86843e03a6195de44b09deda022eec7
|
|
split: test
|
|
type: mteb/cqadupstack-tex
|
|
metrics:
|
|
- type: main_score
|
|
value: 29.707
|
|
- type: map_at_1
|
|
value: 17.159
|
|
- type: map_at_10
|
|
value: 24.869
|
|
- type: map_at_100
|
|
value: 26.021
|
|
- type: map_at_1000
|
|
value: 26.151000000000003
|
|
- type: map_at_20
|
|
value: 25.526
|
|
- type: map_at_3
|
|
value: 22.538
|
|
- type: map_at_5
|
|
value: 23.796999999999997
|
|
- type: mrr_at_1
|
|
value: 20.99105299380592
|
|
- type: mrr_at_10
|
|
value: 28.786336971127096
|
|
- type: mrr_at_100
|
|
value: 29.74490721636805
|
|
- type: mrr_at_1000
|
|
value: 29.823214274100618
|
|
- type: mrr_at_20
|
|
value: 29.363881329195756
|
|
- type: mrr_at_3
|
|
value: 26.531314521679345
|
|
- type: mrr_at_5
|
|
value: 27.7339986235376
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 37.13825685322779
|
|
- type: nauc_map_at_1000_max
|
|
value: 25.949209359787055
|
|
- type: nauc_map_at_1000_std
|
|
value: -0.1789880172036093
|
|
- type: nauc_map_at_100_diff1
|
|
value: 37.13565311027618
|
|
- type: nauc_map_at_100_max
|
|
value: 25.909889375022395
|
|
- type: nauc_map_at_100_std
|
|
value: -0.20169274828783654
|
|
- type: nauc_map_at_10_diff1
|
|
value: 37.412949674073325
|
|
- type: nauc_map_at_10_max
|
|
value: 25.837714322449912
|
|
- type: nauc_map_at_10_std
|
|
value: -0.7989713426808079
|
|
- type: nauc_map_at_1_diff1
|
|
value: 43.66535106611136
|
|
- type: nauc_map_at_1_max
|
|
value: 24.934157845499076
|
|
- type: nauc_map_at_1_std
|
|
value: -2.798761696625911
|
|
- type: nauc_map_at_20_diff1
|
|
value: 37.25188765578179
|
|
- type: nauc_map_at_20_max
|
|
value: 25.887533682661708
|
|
- type: nauc_map_at_20_std
|
|
value: -0.48710070531162597
|
|
- type: nauc_map_at_3_diff1
|
|
value: 38.747478927053876
|
|
- type: nauc_map_at_3_max
|
|
value: 25.551679823476835
|
|
- type: nauc_map_at_3_std
|
|
value: -1.5848393331273871
|
|
- type: nauc_map_at_5_diff1
|
|
value: 38.11902875922142
|
|
- type: nauc_map_at_5_max
|
|
value: 25.84766602647597
|
|
- type: nauc_map_at_5_std
|
|
value: -1.0063039730468788
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 35.409856966860396
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 27.86922067595656
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -0.0734512410447464
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 35.400804471162054
|
|
- type: nauc_mrr_at_100_max
|
|
value: 27.866591373962002
|
|
- type: nauc_mrr_at_100_std
|
|
value: -0.04959722841487173
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 35.5199909370886
|
|
- type: nauc_mrr_at_10_max
|
|
value: 27.962695735822045
|
|
- type: nauc_mrr_at_10_std
|
|
value: -0.5296125062220955
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 41.65630429652082
|
|
- type: nauc_mrr_at_1_max
|
|
value: 27.826862844728982
|
|
- type: nauc_mrr_at_1_std
|
|
value: -2.0718644041769205
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 35.38119545574273
|
|
- type: nauc_mrr_at_20_max
|
|
value: 27.888497220693953
|
|
- type: nauc_mrr_at_20_std
|
|
value: -0.2890434589026467
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 36.603117913849466
|
|
- type: nauc_mrr_at_3_max
|
|
value: 27.947449591583933
|
|
- type: nauc_mrr_at_3_std
|
|
value: -1.0865714056168478
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 35.92459791709931
|
|
- type: nauc_mrr_at_5_max
|
|
value: 28.035251623858272
|
|
- type: nauc_mrr_at_5_std
|
|
value: -0.8711878495606741
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 34.06248430947299
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 26.7997542315953
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 3.3240363708933742
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 33.68748871110203
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 26.362138300414788
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 3.3435049793759717
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 34.50272053437263
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 26.41321919372202
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 0.44722981908997034
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 41.65630429652082
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 27.826862844728982
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -2.0718644041769205
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 34.095928245730065
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 26.278658129351108
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 1.333694029082928
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 36.69705632103637
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 26.78968350373072
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -1.0804397591306258
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 35.72910772416993
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 26.70057707274289
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -0.13486271460127894
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 0.05861252770643225
|
|
- type: nauc_precision_at_1000_max
|
|
value: 18.601946335509112
|
|
- type: nauc_precision_at_1000_std
|
|
value: 9.800060286260463
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 7.363883419620025
|
|
- type: nauc_precision_at_100_max
|
|
value: 22.20848267682575
|
|
- type: nauc_precision_at_100_std
|
|
value: 12.714551550333642
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 21.331506854435275
|
|
- type: nauc_precision_at_10_max
|
|
value: 28.684902701505965
|
|
- type: nauc_precision_at_10_std
|
|
value: 3.6550639959191207
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 41.65630429652082
|
|
- type: nauc_precision_at_1_max
|
|
value: 27.826862844728982
|
|
- type: nauc_precision_at_1_std
|
|
value: -2.0718644041769205
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 16.844978902521646
|
|
- type: nauc_precision_at_20_max
|
|
value: 27.441958887770646
|
|
- type: nauc_precision_at_20_std
|
|
value: 6.3826805047558315
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 30.639398097322594
|
|
- type: nauc_precision_at_3_max
|
|
value: 29.939776959172697
|
|
- type: nauc_precision_at_3_std
|
|
value: -0.20286831584574574
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 26.70376825047474
|
|
- type: nauc_precision_at_5_max
|
|
value: 29.60604358978513
|
|
- type: nauc_precision_at_5_std
|
|
value: 1.5809149742471655
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 17.785715749599042
|
|
- type: nauc_recall_at_1000_max
|
|
value: 23.48376672770539
|
|
- type: nauc_recall_at_1000_std
|
|
value: 30.385000337970858
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 21.05284222570054
|
|
- type: nauc_recall_at_100_max
|
|
value: 21.945063586716614
|
|
- type: nauc_recall_at_100_std
|
|
value: 17.466562038077875
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 26.597231762971674
|
|
- type: nauc_recall_at_10_max
|
|
value: 23.5079436519741
|
|
- type: nauc_recall_at_10_std
|
|
value: 3.263135880492641
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 43.66535106611136
|
|
- type: nauc_recall_at_1_max
|
|
value: 24.934157845499076
|
|
- type: nauc_recall_at_1_std
|
|
value: -2.798761696625911
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 24.832091637143787
|
|
- type: nauc_recall_at_20_max
|
|
value: 22.315764495952237
|
|
- type: nauc_recall_at_20_std
|
|
value: 6.129833251765541
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 32.85408650886733
|
|
- type: nauc_recall_at_3_max
|
|
value: 24.409412121823397
|
|
- type: nauc_recall_at_3_std
|
|
value: 0.04999270761091106
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 30.258414223370007
|
|
- type: nauc_recall_at_5_max
|
|
value: 24.512878195644664
|
|
- type: nauc_recall_at_5_std
|
|
value: 1.849046122226546
|
|
- type: ndcg_at_1
|
|
value: 20.991
|
|
- type: ndcg_at_10
|
|
value: 29.707
|
|
- type: ndcg_at_100
|
|
value: 35.043
|
|
- type: ndcg_at_1000
|
|
value: 38.032
|
|
- type: ndcg_at_20
|
|
value: 31.828
|
|
- type: ndcg_at_3
|
|
value: 25.488
|
|
- type: ndcg_at_5
|
|
value: 27.348
|
|
- type: precision_at_1
|
|
value: 20.991
|
|
- type: precision_at_10
|
|
value: 5.416
|
|
- type: precision_at_100
|
|
value: 0.947
|
|
- type: precision_at_1000
|
|
value: 0.13899999999999998
|
|
- type: precision_at_20
|
|
value: 3.324
|
|
- type: precision_at_3
|
|
value: 12.113
|
|
- type: precision_at_5
|
|
value: 8.734
|
|
- type: recall_at_1
|
|
value: 17.159
|
|
- type: recall_at_10
|
|
value: 40.397
|
|
- type: recall_at_100
|
|
value: 64.139
|
|
- type: recall_at_1000
|
|
value: 85.328
|
|
- type: recall_at_20
|
|
value: 48.193000000000005
|
|
- type: recall_at_3
|
|
value: 28.555999999999997
|
|
- type: recall_at_5
|
|
value: 33.394
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackUnixRetrieval
|
|
revision: 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53
|
|
split: test
|
|
type: mteb/cqadupstack-unix
|
|
metrics:
|
|
- type: main_score
|
|
value: 41.831
|
|
- type: map_at_1
|
|
value: 25.889
|
|
- type: map_at_10
|
|
value: 36.131
|
|
- type: map_at_100
|
|
value: 37.277
|
|
- type: map_at_1000
|
|
value: 37.383
|
|
- type: map_at_20
|
|
value: 36.797000000000004
|
|
- type: map_at_3
|
|
value: 33.194
|
|
- type: map_at_5
|
|
value: 34.88
|
|
- type: mrr_at_1
|
|
value: 30.69029850746269
|
|
- type: mrr_at_10
|
|
value: 40.34274312959011
|
|
- type: mrr_at_100
|
|
value: 41.15568315924076
|
|
- type: mrr_at_1000
|
|
value: 41.21534922643823
|
|
- type: mrr_at_20
|
|
value: 40.81612888073637
|
|
- type: mrr_at_3
|
|
value: 37.624378109452735
|
|
- type: mrr_at_5
|
|
value: 39.25217661691539
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 51.0475973810661
|
|
- type: nauc_map_at_1000_max
|
|
value: 38.75825500903846
|
|
- type: nauc_map_at_1000_std
|
|
value: 1.6136905986292485
|
|
- type: nauc_map_at_100_diff1
|
|
value: 51.04820272616417
|
|
- type: nauc_map_at_100_max
|
|
value: 38.74584044282816
|
|
- type: nauc_map_at_100_std
|
|
value: 1.5969607728429231
|
|
- type: nauc_map_at_10_diff1
|
|
value: 50.94166583581915
|
|
- type: nauc_map_at_10_max
|
|
value: 38.37738102486977
|
|
- type: nauc_map_at_10_std
|
|
value: 1.2635889890868346
|
|
- type: nauc_map_at_1_diff1
|
|
value: 59.242331404755774
|
|
- type: nauc_map_at_1_max
|
|
value: 39.02663876284084
|
|
- type: nauc_map_at_1_std
|
|
value: -0.4739614669668662
|
|
- type: nauc_map_at_20_diff1
|
|
value: 50.97455684751073
|
|
- type: nauc_map_at_20_max
|
|
value: 38.57646135094768
|
|
- type: nauc_map_at_20_std
|
|
value: 1.4640361871795349
|
|
- type: nauc_map_at_3_diff1
|
|
value: 51.608034622903176
|
|
- type: nauc_map_at_3_max
|
|
value: 38.433045221071325
|
|
- type: nauc_map_at_3_std
|
|
value: 0.5831392788488381
|
|
- type: nauc_map_at_5_diff1
|
|
value: 50.947880732714445
|
|
- type: nauc_map_at_5_max
|
|
value: 38.60925399151572
|
|
- type: nauc_map_at_5_std
|
|
value: 1.291960076749259
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 50.210650177335104
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 37.951469256285804
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 0.7902286837699785
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 50.20638219267218
|
|
- type: nauc_mrr_at_100_max
|
|
value: 37.9377948931531
|
|
- type: nauc_mrr_at_100_std
|
|
value: 0.774713370156735
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 49.836111870473935
|
|
- type: nauc_mrr_at_10_max
|
|
value: 37.65348449064669
|
|
- type: nauc_mrr_at_10_std
|
|
value: 0.5231944356104865
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 57.56522049860187
|
|
- type: nauc_mrr_at_1_max
|
|
value: 39.39798439825698
|
|
- type: nauc_mrr_at_1_std
|
|
value: -0.4516317740083426
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 50.1006649557446
|
|
- type: nauc_mrr_at_20_max
|
|
value: 37.84223094800734
|
|
- type: nauc_mrr_at_20_std
|
|
value: 0.8086280885894073
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 50.441725884115996
|
|
- type: nauc_mrr_at_3_max
|
|
value: 37.90807984566849
|
|
- type: nauc_mrr_at_3_std
|
|
value: 0.02550782712808399
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 49.85802035503023
|
|
- type: nauc_mrr_at_5_max
|
|
value: 38.065589153711116
|
|
- type: nauc_mrr_at_5_std
|
|
value: 0.6274639011716443
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 49.03659827838649
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 39.132735746113575
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 3.627422164709519
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 49.00264137357818
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 39.01919928439472
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 3.558699165061359
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 48.26671791603934
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 37.571416815576114
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 1.9403797342170153
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 57.56522049860187
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 39.39798439825698
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -0.4516317740083426
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 48.66105608484808
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 38.22139553816886
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 2.8911511133753782
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 49.00804557017609
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 37.72179482159779
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 0.8400931058476853
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 48.24457268105435
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 38.191301845180604
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 1.9471919379129263
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -12.33889190395623
|
|
- type: nauc_precision_at_1000_max
|
|
value: -1.3115353486004107
|
|
- type: nauc_precision_at_1000_std
|
|
value: 2.495795006465732
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 2.1703067538960084
|
|
- type: nauc_precision_at_100_max
|
|
value: 15.898479441971332
|
|
- type: nauc_precision_at_100_std
|
|
value: 7.910076672658263
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 22.20759907514248
|
|
- type: nauc_precision_at_10_max
|
|
value: 25.51471885225117
|
|
- type: nauc_precision_at_10_std
|
|
value: 2.3609262388624512
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 57.56522049860187
|
|
- type: nauc_precision_at_1_max
|
|
value: 39.39798439825698
|
|
- type: nauc_precision_at_1_std
|
|
value: -0.4516317740083426
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 15.94035009026911
|
|
- type: nauc_precision_at_20_max
|
|
value: 23.178150944386744
|
|
- type: nauc_precision_at_20_std
|
|
value: 5.207387751900332
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 34.99396954995648
|
|
- type: nauc_precision_at_3_max
|
|
value: 33.14418980052923
|
|
- type: nauc_precision_at_3_std
|
|
value: 1.660740116435417
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 29.544849162475362
|
|
- type: nauc_precision_at_5_max
|
|
value: 32.150735196144645
|
|
- type: nauc_precision_at_5_std
|
|
value: 3.323068902360027
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 30.978839058267006
|
|
- type: nauc_recall_at_1000_max
|
|
value: 48.722880061794
|
|
- type: nauc_recall_at_1000_std
|
|
value: 46.28381322993451
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 40.22130846505397
|
|
- type: nauc_recall_at_100_max
|
|
value: 38.28644243336189
|
|
- type: nauc_recall_at_100_std
|
|
value: 15.77321980757386
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 38.9910969333204
|
|
- type: nauc_recall_at_10_max
|
|
value: 32.807008720875984
|
|
- type: nauc_recall_at_10_std
|
|
value: 5.065337152044106
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 59.242331404755774
|
|
- type: nauc_recall_at_1_max
|
|
value: 39.02663876284084
|
|
- type: nauc_recall_at_1_std
|
|
value: -0.4739614669668662
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 40.14875646536079
|
|
- type: nauc_recall_at_20_max
|
|
value: 34.83600129324774
|
|
- type: nauc_recall_at_20_std
|
|
value: 9.01370840232733
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 42.65832338786475
|
|
- type: nauc_recall_at_3_max
|
|
value: 35.56970517818321
|
|
- type: nauc_recall_at_3_std
|
|
value: 1.8050805176967801
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 40.07274624634327
|
|
- type: nauc_recall_at_5_max
|
|
value: 35.74226371272684
|
|
- type: nauc_recall_at_5_std
|
|
value: 4.873290118594757
|
|
- type: ndcg_at_1
|
|
value: 30.69
|
|
- type: ndcg_at_10
|
|
value: 41.831
|
|
- type: ndcg_at_100
|
|
value: 46.966
|
|
- type: ndcg_at_1000
|
|
value: 49.334
|
|
- type: ndcg_at_20
|
|
value: 43.927
|
|
- type: ndcg_at_3
|
|
value: 36.534
|
|
- type: ndcg_at_5
|
|
value: 39.126
|
|
- type: precision_at_1
|
|
value: 30.69
|
|
- type: precision_at_10
|
|
value: 7.1739999999999995
|
|
- type: precision_at_100
|
|
value: 1.095
|
|
- type: precision_at_1000
|
|
value: 0.14200000000000002
|
|
- type: precision_at_20
|
|
value: 4.184
|
|
- type: precision_at_3
|
|
value: 16.853
|
|
- type: precision_at_5
|
|
value: 11.922
|
|
- type: recall_at_1
|
|
value: 25.889
|
|
- type: recall_at_10
|
|
value: 54.962999999999994
|
|
- type: recall_at_100
|
|
value: 77.239
|
|
- type: recall_at_1000
|
|
value: 93.729
|
|
- type: recall_at_20
|
|
value: 62.534
|
|
- type: recall_at_3
|
|
value: 40.336
|
|
- type: recall_at_5
|
|
value: 47.083000000000006
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackWebmastersRetrieval
|
|
revision: 160c094312a0e1facb97e55eeddb698c0abe3571
|
|
split: test
|
|
type: mteb/cqadupstack-webmasters
|
|
metrics:
|
|
- type: main_score
|
|
value: 41.695
|
|
- type: map_at_1
|
|
value: 26.296999999999997
|
|
- type: map_at_10
|
|
value: 35.929
|
|
- type: map_at_100
|
|
value: 37.625
|
|
- type: map_at_1000
|
|
value: 37.856
|
|
- type: map_at_20
|
|
value: 36.831
|
|
- type: map_at_3
|
|
value: 33.042
|
|
- type: map_at_5
|
|
value: 34.552
|
|
- type: mrr_at_1
|
|
value: 31.422924901185773
|
|
- type: mrr_at_10
|
|
value: 40.36718112805069
|
|
- type: mrr_at_100
|
|
value: 41.48635728771627
|
|
- type: mrr_at_1000
|
|
value: 41.53760971899895
|
|
- type: mrr_at_20
|
|
value: 41.05566983667548
|
|
- type: mrr_at_3
|
|
value: 38.24110671936759
|
|
- type: mrr_at_5
|
|
value: 39.49604743083004
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 45.88419073101831
|
|
- type: nauc_map_at_1000_max
|
|
value: 32.272696879964606
|
|
- type: nauc_map_at_1000_std
|
|
value: 6.435633876271509
|
|
- type: nauc_map_at_100_diff1
|
|
value: 46.118272363764085
|
|
- type: nauc_map_at_100_max
|
|
value: 32.459168722724094
|
|
- type: nauc_map_at_100_std
|
|
value: 6.292246088710509
|
|
- type: nauc_map_at_10_diff1
|
|
value: 46.603302676569655
|
|
- type: nauc_map_at_10_max
|
|
value: 32.38318941747706
|
|
- type: nauc_map_at_10_std
|
|
value: 4.720511340512196
|
|
- type: nauc_map_at_1_diff1
|
|
value: 53.474193431022286
|
|
- type: nauc_map_at_1_max
|
|
value: 30.096745684269028
|
|
- type: nauc_map_at_1_std
|
|
value: 0.1635051536400562
|
|
- type: nauc_map_at_20_diff1
|
|
value: 46.25945687266626
|
|
- type: nauc_map_at_20_max
|
|
value: 32.47553839186572
|
|
- type: nauc_map_at_20_std
|
|
value: 5.566329221862548
|
|
- type: nauc_map_at_3_diff1
|
|
value: 47.86679192761851
|
|
- type: nauc_map_at_3_max
|
|
value: 31.531646616728803
|
|
- type: nauc_map_at_3_std
|
|
value: 3.1837781149112496
|
|
- type: nauc_map_at_5_diff1
|
|
value: 46.4585625030729
|
|
- type: nauc_map_at_5_max
|
|
value: 32.013423473733624
|
|
- type: nauc_map_at_5_std
|
|
value: 4.403527966937636
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 44.168029521898646
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 33.231405944995004
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 8.153326593928266
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 44.17027683367582
|
|
- type: nauc_mrr_at_100_max
|
|
value: 33.23422175046355
|
|
- type: nauc_mrr_at_100_std
|
|
value: 8.198284732472755
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 44.2496903067119
|
|
- type: nauc_mrr_at_10_max
|
|
value: 33.055178332856116
|
|
- type: nauc_mrr_at_10_std
|
|
value: 7.831026978775937
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 48.4273290718694
|
|
- type: nauc_mrr_at_1_max
|
|
value: 31.89937877913926
|
|
- type: nauc_mrr_at_1_std
|
|
value: 3.873149993747884
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 44.09113284049905
|
|
- type: nauc_mrr_at_20_max
|
|
value: 33.22019452622306
|
|
- type: nauc_mrr_at_20_std
|
|
value: 8.133802855890329
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 44.86167450862544
|
|
- type: nauc_mrr_at_3_max
|
|
value: 32.98194923216794
|
|
- type: nauc_mrr_at_3_std
|
|
value: 6.9890614678195
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 43.939080994503634
|
|
- type: nauc_mrr_at_5_max
|
|
value: 33.25648484685068
|
|
- type: nauc_mrr_at_5_std
|
|
value: 7.943963197772268
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 43.42006126140444
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 32.89416354016926
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 9.740672987523162
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 43.737763705105145
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 33.102019342275725
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 10.354524698232671
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 43.574979909615
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 32.22335464466024
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 7.827717817165889
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 48.4273290718694
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 31.89937877913926
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 3.873149993747884
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 43.135943873988566
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 32.88264995288679
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 9.104351404942863
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 45.18739397775064
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 31.580166756620283
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 6.137398763080745
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 42.950299500112955
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 32.04130248991469
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 8.322547993875903
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -23.129419591612365
|
|
- type: nauc_precision_at_1000_max
|
|
value: -11.41420275910081
|
|
- type: nauc_precision_at_1000_std
|
|
value: 19.146268912764334
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -12.413671568737618
|
|
- type: nauc_precision_at_100_max
|
|
value: 0.537649304108213
|
|
- type: nauc_precision_at_100_std
|
|
value: 27.325180241816415
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 15.277020606429655
|
|
- type: nauc_precision_at_10_max
|
|
value: 23.51972448360081
|
|
- type: nauc_precision_at_10_std
|
|
value: 19.103862771406927
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 48.4273290718694
|
|
- type: nauc_precision_at_1_max
|
|
value: 31.89937877913926
|
|
- type: nauc_precision_at_1_std
|
|
value: 3.873149993747884
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 4.910626579631313
|
|
- type: nauc_precision_at_20_max
|
|
value: 17.000613397246163
|
|
- type: nauc_precision_at_20_std
|
|
value: 24.370825263718903
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 31.259123635562613
|
|
- type: nauc_precision_at_3_max
|
|
value: 28.91653697836493
|
|
- type: nauc_precision_at_3_std
|
|
value: 11.718828024267332
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 21.896001023343413
|
|
- type: nauc_precision_at_5_max
|
|
value: 26.53717311029016
|
|
- type: nauc_precision_at_5_std
|
|
value: 17.506215861477873
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 15.545423862859614
|
|
- type: nauc_recall_at_1000_max
|
|
value: 33.54097556941026
|
|
- type: nauc_recall_at_1000_std
|
|
value: 41.970927423554926
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 32.29112323650048
|
|
- type: nauc_recall_at_100_max
|
|
value: 31.72353031716839
|
|
- type: nauc_recall_at_100_std
|
|
value: 30.06509939448423
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 36.223842357407875
|
|
- type: nauc_recall_at_10_max
|
|
value: 29.16462133003001
|
|
- type: nauc_recall_at_10_std
|
|
value: 9.404694229411104
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 53.474193431022286
|
|
- type: nauc_recall_at_1_max
|
|
value: 30.096745684269028
|
|
- type: nauc_recall_at_1_std
|
|
value: 0.1635051536400562
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 32.2732032299642
|
|
- type: nauc_recall_at_20_max
|
|
value: 30.699505625402928
|
|
- type: nauc_recall_at_20_std
|
|
value: 15.947782026021681
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 41.746081012759426
|
|
- type: nauc_recall_at_3_max
|
|
value: 29.019436574100016
|
|
- type: nauc_recall_at_3_std
|
|
value: 4.757836484193213
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 35.74337633697516
|
|
- type: nauc_recall_at_5_max
|
|
value: 30.17283125351457
|
|
- type: nauc_recall_at_5_std
|
|
value: 9.488723875013253
|
|
- type: ndcg_at_1
|
|
value: 31.423000000000002
|
|
- type: ndcg_at_10
|
|
value: 41.695
|
|
- type: ndcg_at_100
|
|
value: 48.109
|
|
- type: ndcg_at_1000
|
|
value: 50.39900000000001
|
|
- type: ndcg_at_20
|
|
value: 44.208999999999996
|
|
- type: ndcg_at_3
|
|
value: 37.241
|
|
- type: ndcg_at_5
|
|
value: 39.228
|
|
- type: precision_at_1
|
|
value: 31.423000000000002
|
|
- type: precision_at_10
|
|
value: 7.866
|
|
- type: precision_at_100
|
|
value: 1.603
|
|
- type: precision_at_1000
|
|
value: 0.245
|
|
- type: precision_at_20
|
|
value: 4.99
|
|
- type: precision_at_3
|
|
value: 17.523
|
|
- type: precision_at_5
|
|
value: 12.49
|
|
- type: recall_at_1
|
|
value: 26.296999999999997
|
|
- type: recall_at_10
|
|
value: 52.778000000000006
|
|
- type: recall_at_100
|
|
value: 80.961
|
|
- type: recall_at_1000
|
|
value: 94.894
|
|
- type: recall_at_20
|
|
value: 62.239
|
|
- type: recall_at_3
|
|
value: 39.814
|
|
- type: recall_at_5
|
|
value: 45.381
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CQADupstackWordpressRetrieval
|
|
revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4
|
|
split: test
|
|
type: mteb/cqadupstack-wordpress
|
|
metrics:
|
|
- type: main_score
|
|
value: 33.936
|
|
- type: map_at_1
|
|
value: 21.297
|
|
- type: map_at_10
|
|
value: 29.29
|
|
- type: map_at_100
|
|
value: 30.407
|
|
- type: map_at_1000
|
|
value: 30.514999999999997
|
|
- type: map_at_20
|
|
value: 29.983999999999998
|
|
- type: map_at_3
|
|
value: 26.950000000000003
|
|
- type: map_at_5
|
|
value: 28.287000000000003
|
|
- type: mrr_at_1
|
|
value: 23.65988909426987
|
|
- type: mrr_at_10
|
|
value: 31.57996655224012
|
|
- type: mrr_at_100
|
|
value: 32.58133076268842
|
|
- type: mrr_at_1000
|
|
value: 32.659811204298656
|
|
- type: mrr_at_20
|
|
value: 32.18205959735665
|
|
- type: mrr_at_3
|
|
value: 29.482439926062852
|
|
- type: mrr_at_5
|
|
value: 30.600739371534207
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 33.65655465193916
|
|
- type: nauc_map_at_1000_max
|
|
value: 29.523610574712706
|
|
- type: nauc_map_at_1000_std
|
|
value: -0.48883917163984836
|
|
- type: nauc_map_at_100_diff1
|
|
value: 33.657822812150975
|
|
- type: nauc_map_at_100_max
|
|
value: 29.531870292234302
|
|
- type: nauc_map_at_100_std
|
|
value: -0.49454342105691873
|
|
- type: nauc_map_at_10_diff1
|
|
value: 34.03649741206849
|
|
- type: nauc_map_at_10_max
|
|
value: 29.48133710519135
|
|
- type: nauc_map_at_10_std
|
|
value: -1.3003031064360702
|
|
- type: nauc_map_at_1_diff1
|
|
value: 41.319491034458395
|
|
- type: nauc_map_at_1_max
|
|
value: 30.08436727224079
|
|
- type: nauc_map_at_1_std
|
|
value: -4.283931261517225
|
|
- type: nauc_map_at_20_diff1
|
|
value: 33.644132189750856
|
|
- type: nauc_map_at_20_max
|
|
value: 29.57915168728321
|
|
- type: nauc_map_at_20_std
|
|
value: -0.71252104365507
|
|
- type: nauc_map_at_3_diff1
|
|
value: 33.8965524645013
|
|
- type: nauc_map_at_3_max
|
|
value: 28.898722773976697
|
|
- type: nauc_map_at_3_std
|
|
value: -1.8649217196078969
|
|
- type: nauc_map_at_5_diff1
|
|
value: 33.65177546877711
|
|
- type: nauc_map_at_5_max
|
|
value: 29.449552621308055
|
|
- type: nauc_map_at_5_std
|
|
value: -1.9217932476234898
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 34.21675867856096
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 30.198504997318466
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 0.5352461648974925
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 34.210091539379874
|
|
- type: nauc_mrr_at_100_max
|
|
value: 30.19136090320817
|
|
- type: nauc_mrr_at_100_std
|
|
value: 0.5431068443349623
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 34.50092238629405
|
|
- type: nauc_mrr_at_10_max
|
|
value: 30.360381404088816
|
|
- type: nauc_mrr_at_10_std
|
|
value: 0.007947172236616928
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 41.47500594264137
|
|
- type: nauc_mrr_at_1_max
|
|
value: 30.932862195893563
|
|
- type: nauc_mrr_at_1_std
|
|
value: -3.0060183101242157
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 34.15523281231642
|
|
- type: nauc_mrr_at_20_max
|
|
value: 30.251528444714324
|
|
- type: nauc_mrr_at_20_std
|
|
value: 0.41483749048122587
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 34.54541333351149
|
|
- type: nauc_mrr_at_3_max
|
|
value: 30.357741809442512
|
|
- type: nauc_mrr_at_3_std
|
|
value: -0.5977586679572796
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 34.058033979119465
|
|
- type: nauc_mrr_at_5_max
|
|
value: 30.19093785155445
|
|
- type: nauc_mrr_at_5_std
|
|
value: -0.6829700596355942
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 31.530363860261506
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 29.90327018263153
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 3.033100623143071
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 31.56967408174602
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 29.53643288504651
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 3.4997411689634883
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 32.27374955735248
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 29.519348153684348
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 0.3042011208954651
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 41.47500594264137
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 30.932862195893563
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -3.0060183101242157
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 31.102403306150194
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 29.677553740846967
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 2.1195261321395766
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 32.02333047452249
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 28.888372073027796
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -0.924661397180436
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 31.466174122311667
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 29.307628068867754
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -1.2046829876982417
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -6.075546300902165
|
|
- type: nauc_precision_at_1000_max
|
|
value: -2.187623217222419
|
|
- type: nauc_precision_at_1000_std
|
|
value: 12.584752959282211
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 14.295101079499434
|
|
- type: nauc_precision_at_100_max
|
|
value: 20.388641516894513
|
|
- type: nauc_precision_at_100_std
|
|
value: 21.887960759975524
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 24.536039003837043
|
|
- type: nauc_precision_at_10_max
|
|
value: 29.357326635020637
|
|
- type: nauc_precision_at_10_std
|
|
value: 7.65955284021577
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 41.47500594264137
|
|
- type: nauc_precision_at_1_max
|
|
value: 30.932862195893563
|
|
- type: nauc_precision_at_1_std
|
|
value: -3.0060183101242157
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 18.634308701475955
|
|
- type: nauc_precision_at_20_max
|
|
value: 27.88621903726711
|
|
- type: nauc_precision_at_20_std
|
|
value: 14.96789816785273
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 26.928594601514146
|
|
- type: nauc_precision_at_3_max
|
|
value: 29.653482500006007
|
|
- type: nauc_precision_at_3_std
|
|
value: 2.114869053308719
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 24.137817643228992
|
|
- type: nauc_precision_at_5_max
|
|
value: 29.467809315433215
|
|
- type: nauc_precision_at_5_std
|
|
value: 2.2335268351775777
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 7.561889223723366
|
|
- type: nauc_recall_at_1000_max
|
|
value: 34.64462683484328
|
|
- type: nauc_recall_at_1000_std
|
|
value: 32.07766726976165
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 21.87202458692393
|
|
- type: nauc_recall_at_100_max
|
|
value: 26.060326662408357
|
|
- type: nauc_recall_at_100_std
|
|
value: 20.038540279921996
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 26.59257905849799
|
|
- type: nauc_recall_at_10_max
|
|
value: 27.840231433969887
|
|
- type: nauc_recall_at_10_std
|
|
value: 3.547350776489353
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 41.319491034458395
|
|
- type: nauc_recall_at_1_max
|
|
value: 30.08436727224079
|
|
- type: nauc_recall_at_1_std
|
|
value: -4.283931261517225
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 21.84062118775981
|
|
- type: nauc_recall_at_20_max
|
|
value: 27.960813344120865
|
|
- type: nauc_recall_at_20_std
|
|
value: 9.945117730379264
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 26.240584213234957
|
|
- type: nauc_recall_at_3_max
|
|
value: 27.32563942378109
|
|
- type: nauc_recall_at_3_std
|
|
value: 0.15754039149189397
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 25.327116061029542
|
|
- type: nauc_recall_at_5_max
|
|
value: 28.12294625143933
|
|
- type: nauc_recall_at_5_std
|
|
value: -0.7151467503960333
|
|
- type: ndcg_at_1
|
|
value: 23.66
|
|
- type: ndcg_at_10
|
|
value: 33.936
|
|
- type: ndcg_at_100
|
|
value: 39.172000000000004
|
|
- type: ndcg_at_1000
|
|
value: 41.858000000000004
|
|
- type: ndcg_at_20
|
|
value: 36.248999999999995
|
|
- type: ndcg_at_3
|
|
value: 29.454
|
|
- type: ndcg_at_5
|
|
value: 31.555
|
|
- type: precision_at_1
|
|
value: 23.66
|
|
- type: precision_at_10
|
|
value: 5.323
|
|
- type: precision_at_100
|
|
value: 0.8500000000000001
|
|
- type: precision_at_1000
|
|
value: 0.11900000000000001
|
|
- type: precision_at_20
|
|
value: 3.216
|
|
- type: precision_at_3
|
|
value: 12.568999999999999
|
|
- type: precision_at_5
|
|
value: 8.909
|
|
- type: recall_at_1
|
|
value: 21.297
|
|
- type: recall_at_10
|
|
value: 46.007
|
|
- type: recall_at_100
|
|
value: 69.73700000000001
|
|
- type: recall_at_1000
|
|
value: 89.91900000000001
|
|
- type: recall_at_20
|
|
value: 54.806
|
|
- type: recall_at_3
|
|
value: 33.727000000000004
|
|
- type: recall_at_5
|
|
value: 38.675
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB ClimateFEVER
|
|
revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380
|
|
split: test
|
|
type: mteb/climate-fever
|
|
metrics:
|
|
- type: main_score
|
|
value: 27.794999999999998
|
|
- type: map_at_1
|
|
value: 11.138
|
|
- type: map_at_10
|
|
value: 19.56
|
|
- type: map_at_100
|
|
value: 21.416
|
|
- type: map_at_1000
|
|
value: 21.6
|
|
- type: map_at_20
|
|
value: 20.556
|
|
- type: map_at_3
|
|
value: 16.066
|
|
- type: map_at_5
|
|
value: 17.883
|
|
- type: mrr_at_1
|
|
value: 24.364820846905538
|
|
- type: mrr_at_10
|
|
value: 36.21314823432085
|
|
- type: mrr_at_100
|
|
value: 37.17398469553677
|
|
- type: mrr_at_1000
|
|
value: 37.21013480329614
|
|
- type: mrr_at_20
|
|
value: 36.78840955357927
|
|
- type: mrr_at_3
|
|
value: 32.486427795874
|
|
- type: mrr_at_5
|
|
value: 34.77958740499451
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 22.20775473369687
|
|
- type: nauc_map_at_1000_max
|
|
value: 36.19769366030157
|
|
- type: nauc_map_at_1000_std
|
|
value: 17.568432565671753
|
|
- type: nauc_map_at_100_diff1
|
|
value: 22.202037755951228
|
|
- type: nauc_map_at_100_max
|
|
value: 36.13800341266643
|
|
- type: nauc_map_at_100_std
|
|
value: 17.486248132972992
|
|
- type: nauc_map_at_10_diff1
|
|
value: 22.9042018284273
|
|
- type: nauc_map_at_10_max
|
|
value: 36.08475064127247
|
|
- type: nauc_map_at_10_std
|
|
value: 15.726587888083884
|
|
- type: nauc_map_at_1_diff1
|
|
value: 28.652249616122717
|
|
- type: nauc_map_at_1_max
|
|
value: 32.05131359795648
|
|
- type: nauc_map_at_1_std
|
|
value: 11.262948253532807
|
|
- type: nauc_map_at_20_diff1
|
|
value: 22.451026108322598
|
|
- type: nauc_map_at_20_max
|
|
value: 36.32385371085683
|
|
- type: nauc_map_at_20_std
|
|
value: 16.64337500445571
|
|
- type: nauc_map_at_3_diff1
|
|
value: 23.16011840834893
|
|
- type: nauc_map_at_3_max
|
|
value: 33.24586608916762
|
|
- type: nauc_map_at_3_std
|
|
value: 12.56332091941363
|
|
- type: nauc_map_at_5_diff1
|
|
value: 22.93957941358747
|
|
- type: nauc_map_at_5_max
|
|
value: 34.699460514009
|
|
- type: nauc_map_at_5_std
|
|
value: 14.063661191876298
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 23.24777062437872
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 33.450026215376866
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 20.426349474081853
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 23.23401699847253
|
|
- type: nauc_mrr_at_100_max
|
|
value: 33.45459692613422
|
|
- type: nauc_mrr_at_100_std
|
|
value: 20.440070448958714
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 23.281604083585396
|
|
- type: nauc_mrr_at_10_max
|
|
value: 33.4988527620155
|
|
- type: nauc_mrr_at_10_std
|
|
value: 20.367252947781857
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 26.355328110953575
|
|
- type: nauc_mrr_at_1_max
|
|
value: 30.471508547730092
|
|
- type: nauc_mrr_at_1_std
|
|
value: 16.11568495246132
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 23.140683139461732
|
|
- type: nauc_mrr_at_20_max
|
|
value: 33.48554958878313
|
|
- type: nauc_mrr_at_20_std
|
|
value: 20.44494070529154
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 23.301943271387042
|
|
- type: nauc_mrr_at_3_max
|
|
value: 32.422994068557635
|
|
- type: nauc_mrr_at_3_std
|
|
value: 18.939596173947923
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 23.33922409006143
|
|
- type: nauc_mrr_at_5_max
|
|
value: 33.0752792306208
|
|
- type: nauc_mrr_at_5_std
|
|
value: 19.768166202806604
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 19.998881742304263
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 37.786993391629984
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 24.51994563648888
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 19.98463107036392
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 37.00722603001812
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 23.717744758974426
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 21.62784861661253
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 37.16285223589196
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 19.34332938831155
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 26.355328110953575
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 30.471508547730092
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 16.11568495246132
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 20.55696079927241
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 37.60669992563356
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 21.09713313195671
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 22.08438430773322
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 32.68110059834722
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 15.267429669015595
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 21.715020935808575
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 35.17110301407326
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 16.78243466311895
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -3.2231794613702007
|
|
- type: nauc_precision_at_1000_max
|
|
value: 10.42559310530991
|
|
- type: nauc_precision_at_1000_std
|
|
value: 24.602086786850514
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 1.7021223120345566
|
|
- type: nauc_precision_at_100_max
|
|
value: 17.38852629914526
|
|
- type: nauc_precision_at_100_std
|
|
value: 29.337128327095286
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 12.164922485567033
|
|
- type: nauc_precision_at_10_max
|
|
value: 32.37319082664107
|
|
- type: nauc_precision_at_10_std
|
|
value: 26.300541100072984
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 26.355328110953575
|
|
- type: nauc_precision_at_1_max
|
|
value: 30.471508547730092
|
|
- type: nauc_precision_at_1_std
|
|
value: 16.11568495246132
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 7.385735474290768
|
|
- type: nauc_precision_at_20_max
|
|
value: 28.422173054750115
|
|
- type: nauc_precision_at_20_std
|
|
value: 27.035109636511876
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 16.418314508072836
|
|
- type: nauc_precision_at_3_max
|
|
value: 31.785139366157615
|
|
- type: nauc_precision_at_3_std
|
|
value: 20.32896371836789
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 14.937559885788062
|
|
- type: nauc_precision_at_5_max
|
|
value: 32.24391988837453
|
|
- type: nauc_precision_at_5_std
|
|
value: 23.17707476156323
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 5.616430433184691
|
|
- type: nauc_recall_at_1000_max
|
|
value: 36.55384286718441
|
|
- type: nauc_recall_at_1000_std
|
|
value: 38.50298604014725
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 8.877636292128273
|
|
- type: nauc_recall_at_100_max
|
|
value: 30.860213540250705
|
|
- type: nauc_recall_at_100_std
|
|
value: 28.929321541751467
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 16.07834176997954
|
|
- type: nauc_recall_at_10_max
|
|
value: 35.937627989165364
|
|
- type: nauc_recall_at_10_std
|
|
value: 18.808606461025498
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 28.652249616122717
|
|
- type: nauc_recall_at_1_max
|
|
value: 32.05131359795648
|
|
- type: nauc_recall_at_1_std
|
|
value: 11.262948253532807
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 12.600911526162099
|
|
- type: nauc_recall_at_20_max
|
|
value: 35.177943309574985
|
|
- type: nauc_recall_at_20_std
|
|
value: 21.99092004265232
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 17.49507952659312
|
|
- type: nauc_recall_at_3_max
|
|
value: 31.406559780417105
|
|
- type: nauc_recall_at_3_std
|
|
value: 12.274503076493051
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 16.612956574037305
|
|
- type: nauc_recall_at_5_max
|
|
value: 33.34670088062603
|
|
- type: nauc_recall_at_5_std
|
|
value: 14.445553526736607
|
|
- type: ndcg_at_1
|
|
value: 24.365000000000002
|
|
- type: ndcg_at_10
|
|
value: 27.794999999999998
|
|
- type: ndcg_at_100
|
|
value: 35.11
|
|
- type: ndcg_at_1000
|
|
value: 38.383
|
|
- type: ndcg_at_20
|
|
value: 30.616
|
|
- type: ndcg_at_3
|
|
value: 21.97
|
|
- type: ndcg_at_5
|
|
value: 24.264
|
|
- type: precision_at_1
|
|
value: 24.365000000000002
|
|
- type: precision_at_10
|
|
value: 8.827
|
|
- type: precision_at_100
|
|
value: 1.6660000000000001
|
|
- type: precision_at_1000
|
|
value: 0.22799999999999998
|
|
- type: precision_at_20
|
|
value: 5.6160000000000005
|
|
- type: precision_at_3
|
|
value: 16.2
|
|
- type: precision_at_5
|
|
value: 13.055
|
|
- type: recall_at_1
|
|
value: 11.138
|
|
- type: recall_at_10
|
|
value: 34.454
|
|
- type: recall_at_100
|
|
value: 59.648
|
|
- type: recall_at_1000
|
|
value: 77.823
|
|
- type: recall_at_20
|
|
value: 42.476
|
|
- type: recall_at_3
|
|
value: 20.630000000000003
|
|
- type: recall_at_5
|
|
value: 26.517000000000003
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB DBPedia
|
|
revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659
|
|
split: test
|
|
type: mteb/dbpedia
|
|
metrics:
|
|
- type: main_score
|
|
value: 39.582
|
|
- type: map_at_1
|
|
value: 8.193
|
|
- type: map_at_10
|
|
value: 18.838
|
|
- type: map_at_100
|
|
value: 26.791999999999998
|
|
- type: map_at_1000
|
|
value: 28.659000000000002
|
|
- type: map_at_20
|
|
value: 21.678
|
|
- type: map_at_3
|
|
value: 13.535
|
|
- type: map_at_5
|
|
value: 15.706000000000001
|
|
- type: mrr_at_1
|
|
value: 61.25000000000001
|
|
- type: mrr_at_10
|
|
value: 71.5827380952381
|
|
- type: mrr_at_100
|
|
value: 71.92227940484834
|
|
- type: mrr_at_1000
|
|
value: 71.92843656364919
|
|
- type: mrr_at_20
|
|
value: 71.82391254578756
|
|
- type: mrr_at_3
|
|
value: 69.54166666666667
|
|
- type: mrr_at_5
|
|
value: 70.89166666666667
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 20.81525104511085
|
|
- type: nauc_map_at_1000_max
|
|
value: 12.28738487676873
|
|
- type: nauc_map_at_1000_std
|
|
value: 24.87551199629768
|
|
- type: nauc_map_at_100_diff1
|
|
value: 21.693837182713217
|
|
- type: nauc_map_at_100_max
|
|
value: 8.69725977707396
|
|
- type: nauc_map_at_100_std
|
|
value: 21.354633072475515
|
|
- type: nauc_map_at_10_diff1
|
|
value: 24.388731902741767
|
|
- type: nauc_map_at_10_max
|
|
value: -4.0866423282629585
|
|
- type: nauc_map_at_10_std
|
|
value: -0.9510081645949322
|
|
- type: nauc_map_at_1_diff1
|
|
value: 32.58191575261803
|
|
- type: nauc_map_at_1_max
|
|
value: -10.57813486927926
|
|
- type: nauc_map_at_1_std
|
|
value: -9.588423425329879
|
|
- type: nauc_map_at_20_diff1
|
|
value: 24.050743021827124
|
|
- type: nauc_map_at_20_max
|
|
value: 0.6686240161106345
|
|
- type: nauc_map_at_20_std
|
|
value: 6.53795559839344
|
|
- type: nauc_map_at_3_diff1
|
|
value: 26.43827919066607
|
|
- type: nauc_map_at_3_max
|
|
value: -10.727017270257825
|
|
- type: nauc_map_at_3_std
|
|
value: -9.512078389268677
|
|
- type: nauc_map_at_5_diff1
|
|
value: 25.71002404847907
|
|
- type: nauc_map_at_5_max
|
|
value: -7.097015507701878
|
|
- type: nauc_map_at_5_std
|
|
value: -6.476602516100202
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 45.608034553728835
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 30.922028122514266
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 34.21750207725521
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 45.590642197534805
|
|
- type: nauc_mrr_at_100_max
|
|
value: 30.930031708368194
|
|
- type: nauc_mrr_at_100_std
|
|
value: 34.21945637610545
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 45.540994123130126
|
|
- type: nauc_mrr_at_10_max
|
|
value: 30.83734303048343
|
|
- type: nauc_mrr_at_10_std
|
|
value: 34.348404162478694
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 49.560483335546415
|
|
- type: nauc_mrr_at_1_max
|
|
value: 28.883661816871232
|
|
- type: nauc_mrr_at_1_std
|
|
value: 30.89553654418874
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 45.499322734057515
|
|
- type: nauc_mrr_at_20_max
|
|
value: 30.918972161205733
|
|
- type: nauc_mrr_at_20_std
|
|
value: 34.282904222510595
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 45.39622724954005
|
|
- type: nauc_mrr_at_3_max
|
|
value: 31.457074078677454
|
|
- type: nauc_mrr_at_3_std
|
|
value: 34.079043384571555
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 44.71358730464237
|
|
- type: nauc_mrr_at_5_max
|
|
value: 30.69295376764748
|
|
- type: nauc_mrr_at_5_std
|
|
value: 34.31128800389916
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 23.109017019057422
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 23.08462483716398
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 36.8911815972109
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 23.827280037818173
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 13.309666633249211
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 28.44384667395871
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 26.972856731999386
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 14.620707258357266
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 23.111341368346462
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 43.59088178770794
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 21.904917923054317
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 21.98647522718905
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 26.283361626051914
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 11.10518046266052
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 21.355473505613944
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 30.024148446672083
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 18.48737788479935
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 23.24967559220411
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 27.31687195788342
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 17.233426051712428
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 22.98467702068255
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -13.448141290306074
|
|
- type: nauc_precision_at_1000_max
|
|
value: 42.26049965587544
|
|
- type: nauc_precision_at_1000_std
|
|
value: 17.838997647650835
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -5.070670934466766
|
|
- type: nauc_precision_at_100_max
|
|
value: 33.96276536553548
|
|
- type: nauc_precision_at_100_std
|
|
value: 47.592571562595765
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 5.079452111840327
|
|
- type: nauc_precision_at_10_max
|
|
value: 33.145301874068146
|
|
- type: nauc_precision_at_10_std
|
|
value: 46.26256386765269
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 49.560483335546415
|
|
- type: nauc_precision_at_1_max
|
|
value: 28.883661816871232
|
|
- type: nauc_precision_at_1_std
|
|
value: 30.89553654418874
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 3.253674888888517
|
|
- type: nauc_precision_at_20_max
|
|
value: 34.667104498369575
|
|
- type: nauc_precision_at_20_std
|
|
value: 49.202859485875535
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 15.790066053828234
|
|
- type: nauc_precision_at_3_max
|
|
value: 27.215083484496542
|
|
- type: nauc_precision_at_3_std
|
|
value: 33.11505410450215
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 9.530674873702113
|
|
- type: nauc_precision_at_5_max
|
|
value: 31.21998248355014
|
|
- type: nauc_precision_at_5_std
|
|
value: 39.07247161423012
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 5.70231960458697
|
|
- type: nauc_recall_at_1000_max
|
|
value: 16.173798281531525
|
|
- type: nauc_recall_at_1000_std
|
|
value: 40.45772368713694
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 9.815485122352673
|
|
- type: nauc_recall_at_100_max
|
|
value: 3.5894004884530735
|
|
- type: nauc_recall_at_100_std
|
|
value: 23.442799836302864
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 14.537879655467389
|
|
- type: nauc_recall_at_10_max
|
|
value: -10.56087357341994
|
|
- type: nauc_recall_at_10_std
|
|
value: -7.372934296480146
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 32.58191575261803
|
|
- type: nauc_recall_at_1_max
|
|
value: -10.57813486927926
|
|
- type: nauc_recall_at_1_std
|
|
value: -9.588423425329879
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 13.359604621352824
|
|
- type: nauc_recall_at_20_max
|
|
value: -6.037674048018859
|
|
- type: nauc_recall_at_20_std
|
|
value: -0.191231970406073
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 20.620776298724362
|
|
- type: nauc_recall_at_3_max
|
|
value: -14.34692846751201
|
|
- type: nauc_recall_at_3_std
|
|
value: -12.202460021792232
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 17.573424943863706
|
|
- type: nauc_recall_at_5_max
|
|
value: -10.968843043485661
|
|
- type: nauc_recall_at_5_std
|
|
value: -10.513373048008399
|
|
- type: ndcg_at_1
|
|
value: 48.375
|
|
- type: ndcg_at_10
|
|
value: 39.582
|
|
- type: ndcg_at_100
|
|
value: 45.259
|
|
- type: ndcg_at_1000
|
|
value: 53.022000000000006
|
|
- type: ndcg_at_20
|
|
value: 39.038000000000004
|
|
- type: ndcg_at_3
|
|
value: 42.802
|
|
- type: ndcg_at_5
|
|
value: 40.538000000000004
|
|
- type: precision_at_1
|
|
value: 61.25000000000001
|
|
- type: precision_at_10
|
|
value: 32.2
|
|
- type: precision_at_100
|
|
value: 10.545
|
|
- type: precision_at_1000
|
|
value: 2.2880000000000003
|
|
- type: precision_at_20
|
|
value: 24.05
|
|
- type: precision_at_3
|
|
value: 48.083
|
|
- type: precision_at_5
|
|
value: 40.65
|
|
- type: recall_at_1
|
|
value: 8.193
|
|
- type: recall_at_10
|
|
value: 25.519
|
|
- type: recall_at_100
|
|
value: 54.124
|
|
- type: recall_at_1000
|
|
value: 77.92099999999999
|
|
- type: recall_at_20
|
|
value: 32.385999999999996
|
|
- type: recall_at_3
|
|
value: 15.211
|
|
- type: recall_at_5
|
|
value: 18.891
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB EmotionClassification
|
|
revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
|
|
split: test
|
|
type: mteb/emotion
|
|
metrics:
|
|
- type: accuracy
|
|
value: 85.565
|
|
- type: f1
|
|
value: 81.12346731656551
|
|
- type: f1_weighted
|
|
value: 85.98372374550102
|
|
- type: main_score
|
|
value: 85.565
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB FEVER
|
|
revision: bea83ef9e8fb933d90a2f1d5515737465d613e12
|
|
split: test
|
|
type: mteb/fever
|
|
metrics:
|
|
- type: main_score
|
|
value: 86.026
|
|
- type: map_at_1
|
|
value: 73.339
|
|
- type: map_at_10
|
|
value: 81.943
|
|
- type: map_at_100
|
|
value: 82.12899999999999
|
|
- type: map_at_1000
|
|
value: 82.145
|
|
- type: map_at_20
|
|
value: 82.05799999999999
|
|
- type: map_at_3
|
|
value: 80.827
|
|
- type: map_at_5
|
|
value: 81.628
|
|
- type: mrr_at_1
|
|
value: 78.98289828982898
|
|
- type: mrr_at_10
|
|
value: 87.04412703175062
|
|
- type: mrr_at_100
|
|
value: 87.1023996343652
|
|
- type: mrr_at_1000
|
|
value: 87.10370910386118
|
|
- type: mrr_at_20
|
|
value: 87.08615223713309
|
|
- type: mrr_at_3
|
|
value: 86.2386238623861
|
|
- type: mrr_at_5
|
|
value: 86.86568656865666
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 48.22616948132843
|
|
- type: nauc_map_at_1000_max
|
|
value: 1.6340021380561394
|
|
- type: nauc_map_at_1000_std
|
|
value: -25.200746351372793
|
|
- type: nauc_map_at_100_diff1
|
|
value: 48.198187398812806
|
|
- type: nauc_map_at_100_max
|
|
value: 1.6220191408228601
|
|
- type: nauc_map_at_100_std
|
|
value: -25.193042721137566
|
|
- type: nauc_map_at_10_diff1
|
|
value: 48.00585391806132
|
|
- type: nauc_map_at_10_max
|
|
value: 1.4817376575907626
|
|
- type: nauc_map_at_10_std
|
|
value: -25.201484788329843
|
|
- type: nauc_map_at_1_diff1
|
|
value: 52.76212226788538
|
|
- type: nauc_map_at_1_max
|
|
value: 0.0520314144507071
|
|
- type: nauc_map_at_1_std
|
|
value: -26.20833932232049
|
|
- type: nauc_map_at_20_diff1
|
|
value: 48.12533777970878
|
|
- type: nauc_map_at_20_max
|
|
value: 1.5240294773565493
|
|
- type: nauc_map_at_20_std
|
|
value: -25.192450618181123
|
|
- type: nauc_map_at_3_diff1
|
|
value: 47.96480519565094
|
|
- type: nauc_map_at_3_max
|
|
value: 1.1887774816136902
|
|
- type: nauc_map_at_3_std
|
|
value: -26.31363833371711
|
|
- type: nauc_map_at_5_diff1
|
|
value: 47.79080333430883
|
|
- type: nauc_map_at_5_max
|
|
value: 1.6220551876503297
|
|
- type: nauc_map_at_5_std
|
|
value: -25.250585439913415
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 64.95992140968579
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 1.6737288643493216
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -37.732249646223224
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 64.95845005240741
|
|
- type: nauc_mrr_at_100_max
|
|
value: 1.6807060884331666
|
|
- type: nauc_mrr_at_100_std
|
|
value: -37.73314881154047
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 64.9115307834577
|
|
- type: nauc_mrr_at_10_max
|
|
value: 1.7195209183889257
|
|
- type: nauc_mrr_at_10_std
|
|
value: -37.88536525017639
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 66.13713227430745
|
|
- type: nauc_mrr_at_1_max
|
|
value: 0.37082095312916874
|
|
- type: nauc_mrr_at_1_std
|
|
value: -34.379038222842254
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 64.95488651854674
|
|
- type: nauc_mrr_at_20_max
|
|
value: 1.6985375216432168
|
|
- type: nauc_mrr_at_20_std
|
|
value: -37.755703989608705
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 64.9535677343948
|
|
- type: nauc_mrr_at_3_max
|
|
value: 1.5195414353630512
|
|
- type: nauc_mrr_at_3_std
|
|
value: -39.21735562852805
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 64.85513437757459
|
|
- type: nauc_mrr_at_5_max
|
|
value: 1.9382830256224208
|
|
- type: nauc_mrr_at_5_std
|
|
value: -38.043842104083545
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 49.74095915307536
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 2.605169283095937
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -25.835814259340832
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 49.002859024867945
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 2.5116469969385884
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -25.479921013562272
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 48.25197176801494
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 1.9108104946028264
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -25.780784974391295
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 66.13713227430745
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 0.37082095312916874
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -34.379038222842254
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 48.59674729644139
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 1.9950884849133927
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -25.569135598052622
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 49.305511135576275
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 1.8638668857901368
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -29.02269314595723
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 48.1680764938404
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 2.4842182285117964
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -26.244542780767375
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -4.478420343136971
|
|
- type: nauc_precision_at_1000_max
|
|
value: 11.70949232501659
|
|
- type: nauc_precision_at_1000_std
|
|
value: 1.7386198733671119
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -4.172269763651759
|
|
- type: nauc_precision_at_100_max
|
|
value: 13.082661117154743
|
|
- type: nauc_precision_at_100_std
|
|
value: 1.8002212793127355
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 5.702289274109695
|
|
- type: nauc_precision_at_10_max
|
|
value: 8.484620250928458
|
|
- type: nauc_precision_at_10_std
|
|
value: -8.132389694515703
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 66.13713227430745
|
|
- type: nauc_precision_at_1_max
|
|
value: 0.37082095312916874
|
|
- type: nauc_precision_at_1_std
|
|
value: -34.379038222842254
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 0.5564831263316283
|
|
- type: nauc_precision_at_20_max
|
|
value: 8.881191911131173
|
|
- type: nauc_precision_at_20_std
|
|
value: -3.696180671957281
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 35.75913314270679
|
|
- type: nauc_precision_at_3_max
|
|
value: 7.896253718358011
|
|
- type: nauc_precision_at_3_std
|
|
value: -33.8336411888768
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 17.101795422527648
|
|
- type: nauc_precision_at_5_max
|
|
value: 11.993885038446976
|
|
- type: nauc_precision_at_5_std
|
|
value: -16.39044303210142
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 1.765610982286282
|
|
- type: nauc_recall_at_1000_max
|
|
value: 16.0490507693684
|
|
- type: nauc_recall_at_1000_std
|
|
value: 28.474043694387696
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 6.2725603406909265
|
|
- type: nauc_recall_at_100_max
|
|
value: 10.665282199745704
|
|
- type: nauc_recall_at_100_std
|
|
value: 13.266482323582757
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 16.010002473322103
|
|
- type: nauc_recall_at_10_max
|
|
value: 4.051158641772395
|
|
- type: nauc_recall_at_10_std
|
|
value: -3.963886778602456
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 52.76212226788538
|
|
- type: nauc_recall_at_1_max
|
|
value: 0.0520314144507071
|
|
- type: nauc_recall_at_1_std
|
|
value: -26.20833932232049
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 12.763325751516286
|
|
- type: nauc_recall_at_20_max
|
|
value: 4.589618045061225
|
|
- type: nauc_recall_at_20_std
|
|
value: 2.3135711002947525
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 31.878202992328298
|
|
- type: nauc_recall_at_3_max
|
|
value: 2.398044119809843
|
|
- type: nauc_recall_at_3_std
|
|
value: -22.48228292127779
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 22.01091185405021
|
|
- type: nauc_recall_at_5_max
|
|
value: 6.161863454884261
|
|
- type: nauc_recall_at_5_std
|
|
value: -10.442113305092082
|
|
- type: ndcg_at_1
|
|
value: 78.983
|
|
- type: ndcg_at_10
|
|
value: 86.026
|
|
- type: ndcg_at_100
|
|
value: 86.666
|
|
- type: ndcg_at_1000
|
|
value: 86.945
|
|
- type: ndcg_at_20
|
|
value: 86.333
|
|
- type: ndcg_at_3
|
|
value: 84.269
|
|
- type: ndcg_at_5
|
|
value: 85.439
|
|
- type: precision_at_1
|
|
value: 78.983
|
|
- type: precision_at_10
|
|
value: 10.282
|
|
- type: precision_at_100
|
|
value: 1.078
|
|
- type: precision_at_1000
|
|
value: 0.11199999999999999
|
|
- type: precision_at_20
|
|
value: 5.2330000000000005
|
|
- type: precision_at_3
|
|
value: 32.218
|
|
- type: precision_at_5
|
|
value: 20.06
|
|
- type: recall_at_1
|
|
value: 73.339
|
|
- type: recall_at_10
|
|
value: 93.557
|
|
- type: recall_at_100
|
|
value: 96.03399999999999
|
|
- type: recall_at_1000
|
|
value: 97.784
|
|
- type: recall_at_20
|
|
value: 94.6
|
|
- type: recall_at_3
|
|
value: 88.851
|
|
- type: recall_at_5
|
|
value: 91.81
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB FiQA2018
|
|
revision: 27a168819829fe9bcd655c2df245fb19452e8e06
|
|
split: test
|
|
type: mteb/fiqa
|
|
metrics:
|
|
- type: main_score
|
|
value: 45.019
|
|
- type: map_at_1
|
|
value: 21.923000000000002
|
|
- type: map_at_10
|
|
value: 36.661
|
|
- type: map_at_100
|
|
value: 38.727000000000004
|
|
- type: map_at_1000
|
|
value: 38.896
|
|
- type: map_at_20
|
|
value: 37.821
|
|
- type: map_at_3
|
|
value: 31.812
|
|
- type: map_at_5
|
|
value: 34.474
|
|
- type: mrr_at_1
|
|
value: 43.05555555555556
|
|
- type: mrr_at_10
|
|
value: 52.714824612972755
|
|
- type: mrr_at_100
|
|
value: 53.47543808894285
|
|
- type: mrr_at_1000
|
|
value: 53.50616025822894
|
|
- type: mrr_at_20
|
|
value: 53.14543059863263
|
|
- type: mrr_at_3
|
|
value: 50.10288065843621
|
|
- type: mrr_at_5
|
|
value: 51.715534979423836
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 41.776705312708486
|
|
- type: nauc_map_at_1000_max
|
|
value: 24.93532754336337
|
|
- type: nauc_map_at_1000_std
|
|
value: -2.794190590614799
|
|
- type: nauc_map_at_100_diff1
|
|
value: 41.73579109673881
|
|
- type: nauc_map_at_100_max
|
|
value: 24.80625280860252
|
|
- type: nauc_map_at_100_std
|
|
value: -2.814441295874619
|
|
- type: nauc_map_at_10_diff1
|
|
value: 41.75395538260581
|
|
- type: nauc_map_at_10_max
|
|
value: 23.219207680303324
|
|
- type: nauc_map_at_10_std
|
|
value: -3.5779070328036138
|
|
- type: nauc_map_at_1_diff1
|
|
value: 48.46545399169614
|
|
- type: nauc_map_at_1_max
|
|
value: 16.49315969594624
|
|
- type: nauc_map_at_1_std
|
|
value: -7.505787454483636
|
|
- type: nauc_map_at_20_diff1
|
|
value: 41.53641801097531
|
|
- type: nauc_map_at_20_max
|
|
value: 24.00770569213574
|
|
- type: nauc_map_at_20_std
|
|
value: -3.191754163523877
|
|
- type: nauc_map_at_3_diff1
|
|
value: 41.75052616046243
|
|
- type: nauc_map_at_3_max
|
|
value: 19.115081667001014
|
|
- type: nauc_map_at_3_std
|
|
value: -6.668596004487064
|
|
- type: nauc_map_at_5_diff1
|
|
value: 42.45446754604312
|
|
- type: nauc_map_at_5_max
|
|
value: 20.947253345126185
|
|
- type: nauc_map_at_5_std
|
|
value: -5.125992439200763
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 52.09717084990717
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 38.086957556354456
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -0.68079244284855
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 52.081504543550686
|
|
- type: nauc_mrr_at_100_max
|
|
value: 38.10189737899758
|
|
- type: nauc_mrr_at_100_std
|
|
value: -0.6731400759499799
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 51.962775327926934
|
|
- type: nauc_mrr_at_10_max
|
|
value: 37.860734658269976
|
|
- type: nauc_mrr_at_10_std
|
|
value: -0.8627588620266099
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 56.643374967422865
|
|
- type: nauc_mrr_at_1_max
|
|
value: 37.424164231372195
|
|
- type: nauc_mrr_at_1_std
|
|
value: -3.808604224746232
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 51.9634718440668
|
|
- type: nauc_mrr_at_20_max
|
|
value: 37.99992134394818
|
|
- type: nauc_mrr_at_20_std
|
|
value: -0.5725435512805715
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 51.9083290591896
|
|
- type: nauc_mrr_at_3_max
|
|
value: 37.49495462369628
|
|
- type: nauc_mrr_at_3_std
|
|
value: -2.193915400523023
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 52.24074329239152
|
|
- type: nauc_mrr_at_5_max
|
|
value: 37.96365352861984
|
|
- type: nauc_mrr_at_5_std
|
|
value: -1.5116002789297864
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 43.88564426048843
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 31.371070838376326
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 1.182058822041445
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 43.47882005622348
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 30.23626893448966
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 1.3554256181078206
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 42.78328747987686
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 26.971284497406334
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -0.9361763271905158
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 56.643374967422865
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 37.424164231372195
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -3.808604224746232
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 42.51200178317055
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 27.807479427212844
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -0.16279719845344157
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 41.983935082179556
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 28.446235814415143
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -3.0007943000595003
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 43.21852196702825
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 26.601248066336986
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -2.5471886292781702
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -0.26010199321259797
|
|
- type: nauc_precision_at_1000_max
|
|
value: 35.79601474558423
|
|
- type: nauc_precision_at_1000_std
|
|
value: 14.342818001909988
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 6.004698224173632
|
|
- type: nauc_precision_at_100_max
|
|
value: 38.52857855255943
|
|
- type: nauc_precision_at_100_std
|
|
value: 16.21705591642149
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 17.49728453546782
|
|
- type: nauc_precision_at_10_max
|
|
value: 38.24671033647839
|
|
- type: nauc_precision_at_10_std
|
|
value: 12.030940471652098
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 56.643374967422865
|
|
- type: nauc_precision_at_1_max
|
|
value: 37.424164231372195
|
|
- type: nauc_precision_at_1_std
|
|
value: -3.808604224746232
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 13.057739432783794
|
|
- type: nauc_precision_at_20_max
|
|
value: 37.84177604877064
|
|
- type: nauc_precision_at_20_std
|
|
value: 13.135243737603359
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 29.106393446078787
|
|
- type: nauc_precision_at_3_max
|
|
value: 33.51402929333319
|
|
- type: nauc_precision_at_3_std
|
|
value: 1.9298573035534488
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 25.039378213923403
|
|
- type: nauc_precision_at_5_max
|
|
value: 36.213261098065125
|
|
- type: nauc_precision_at_5_std
|
|
value: 7.142334933169122
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 24.897608581023757
|
|
- type: nauc_recall_at_1000_max
|
|
value: 24.60932291382376
|
|
- type: nauc_recall_at_1000_std
|
|
value: 30.05990115014322
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 30.807527684131564
|
|
- type: nauc_recall_at_100_max
|
|
value: 22.540558835740985
|
|
- type: nauc_recall_at_100_std
|
|
value: 14.493739358980907
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 31.683742260409076
|
|
- type: nauc_recall_at_10_max
|
|
value: 17.828711448272134
|
|
- type: nauc_recall_at_10_std
|
|
value: 1.899605838015785
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 48.46545399169614
|
|
- type: nauc_recall_at_1_max
|
|
value: 16.49315969594624
|
|
- type: nauc_recall_at_1_std
|
|
value: -7.505787454483636
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 30.08305577595204
|
|
- type: nauc_recall_at_20_max
|
|
value: 18.75062281011906
|
|
- type: nauc_recall_at_20_std
|
|
value: 4.502661433146342
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 33.53153516576839
|
|
- type: nauc_recall_at_3_max
|
|
value: 14.790607412204485
|
|
- type: nauc_recall_at_3_std
|
|
value: -6.1140323409194846
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 35.64279484984148
|
|
- type: nauc_recall_at_5_max
|
|
value: 15.401875599379574
|
|
- type: nauc_recall_at_5_std
|
|
value: -3.2844856697915774
|
|
- type: ndcg_at_1
|
|
value: 43.056
|
|
- type: ndcg_at_10
|
|
value: 45.019
|
|
- type: ndcg_at_100
|
|
value: 51.98199999999999
|
|
- type: ndcg_at_1000
|
|
value: 54.581999999999994
|
|
- type: ndcg_at_20
|
|
value: 47.721999999999994
|
|
- type: ndcg_at_3
|
|
value: 40.54
|
|
- type: ndcg_at_5
|
|
value: 42.142
|
|
- type: precision_at_1
|
|
value: 43.056
|
|
- type: precision_at_10
|
|
value: 12.531
|
|
- type: precision_at_100
|
|
value: 1.9949999999999999
|
|
- type: precision_at_1000
|
|
value: 0.245
|
|
- type: precision_at_20
|
|
value: 7.446
|
|
- type: precision_at_3
|
|
value: 27.058
|
|
- type: precision_at_5
|
|
value: 20.061999999999998
|
|
- type: recall_at_1
|
|
value: 21.923000000000002
|
|
- type: recall_at_10
|
|
value: 52.85300000000001
|
|
- type: recall_at_100
|
|
value: 78.133
|
|
- type: recall_at_1000
|
|
value: 93.75
|
|
- type: recall_at_20
|
|
value: 61.085
|
|
- type: recall_at_3
|
|
value: 37.118
|
|
- type: recall_at_5
|
|
value: 44.031
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB HotpotQA
|
|
revision: ab518f4d6fcca38d87c25209f94beba119d02014
|
|
split: test
|
|
type: mteb/hotpotqa
|
|
metrics:
|
|
- type: main_score
|
|
value: 68.65299999999999
|
|
- type: map_at_1
|
|
value: 38.893
|
|
- type: map_at_10
|
|
value: 59.375
|
|
- type: map_at_100
|
|
value: 60.303
|
|
- type: map_at_1000
|
|
value: 60.364
|
|
- type: map_at_20
|
|
value: 59.964
|
|
- type: map_at_3
|
|
value: 55.718
|
|
- type: map_at_5
|
|
value: 57.99999999999999
|
|
- type: mrr_at_1
|
|
value: 77.78528021607022
|
|
- type: mrr_at_10
|
|
value: 84.49470006323453
|
|
- type: mrr_at_100
|
|
value: 84.6519637218647
|
|
- type: mrr_at_1000
|
|
value: 84.65768034160618
|
|
- type: mrr_at_20
|
|
value: 84.61055874712832
|
|
- type: mrr_at_3
|
|
value: 83.59441818591013
|
|
- type: mrr_at_5
|
|
value: 84.19266261534956
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 15.948378928650673
|
|
- type: nauc_map_at_1000_max
|
|
value: 15.711635353869994
|
|
- type: nauc_map_at_1000_std
|
|
value: 0.937019577383957
|
|
- type: nauc_map_at_100_diff1
|
|
value: 15.918426215773247
|
|
- type: nauc_map_at_100_max
|
|
value: 15.699627284031124
|
|
- type: nauc_map_at_100_std
|
|
value: 0.9584857374941618
|
|
- type: nauc_map_at_10_diff1
|
|
value: 15.879270822613408
|
|
- type: nauc_map_at_10_max
|
|
value: 15.463063162099125
|
|
- type: nauc_map_at_10_std
|
|
value: 0.15481877422177437
|
|
- type: nauc_map_at_1_diff1
|
|
value: 71.30652188008001
|
|
- type: nauc_map_at_1_max
|
|
value: 32.60802008342313
|
|
- type: nauc_map_at_1_std
|
|
value: -12.29496015891874
|
|
- type: nauc_map_at_20_diff1
|
|
value: 15.853758892635852
|
|
- type: nauc_map_at_20_max
|
|
value: 15.570900027569573
|
|
- type: nauc_map_at_20_std
|
|
value: 0.6783433634347852
|
|
- type: nauc_map_at_3_diff1
|
|
value: 17.97014394473015
|
|
- type: nauc_map_at_3_max
|
|
value: 15.218485551181926
|
|
- type: nauc_map_at_3_std
|
|
value: -2.4303445320319272
|
|
- type: nauc_map_at_5_diff1
|
|
value: 16.50404017618271
|
|
- type: nauc_map_at_5_max
|
|
value: 15.285663669100073
|
|
- type: nauc_map_at_5_std
|
|
value: -0.989351556289713
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 70.0763435325149
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 34.01106818267054
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -10.558570244805534
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 70.0763826742575
|
|
- type: nauc_mrr_at_100_max
|
|
value: 34.01329127860268
|
|
- type: nauc_mrr_at_100_std
|
|
value: -10.553859035770314
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 70.03690200308235
|
|
- type: nauc_mrr_at_10_max
|
|
value: 34.10786779680883
|
|
- type: nauc_mrr_at_10_std
|
|
value: -10.509981664609755
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 71.30652188008001
|
|
- type: nauc_mrr_at_1_max
|
|
value: 32.60802008342313
|
|
- type: nauc_mrr_at_1_std
|
|
value: -12.29496015891874
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 70.07320564989382
|
|
- type: nauc_mrr_at_20_max
|
|
value: 34.01911070550699
|
|
- type: nauc_mrr_at_20_std
|
|
value: -10.532501476325248
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 69.73518331018965
|
|
- type: nauc_mrr_at_3_max
|
|
value: 33.7438084424745
|
|
- type: nauc_mrr_at_3_std
|
|
value: -11.302692900313119
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 69.86565354847778
|
|
- type: nauc_mrr_at_5_max
|
|
value: 34.135593857390504
|
|
- type: nauc_mrr_at_5_std
|
|
value: -10.380178093077621
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 20.865436555566845
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 18.83121871269731
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 3.566623532300052
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 19.90357263881322
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 18.387111355628193
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 4.243680531655493
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 19.721051339510907
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 17.558512453515227
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 1.2891095080720567
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 71.30652188008001
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 32.60802008342313
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -12.29496015891874
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 19.519425870891023
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 17.77152674804043
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 2.7253915106561712
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 23.595619290089495
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 17.443501928111456
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -3.1185231896019183
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 21.128676475251222
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 17.427440887891148
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -0.8006655617871765
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -18.605360521020412
|
|
- type: nauc_precision_at_1000_max
|
|
value: 13.992651128348118
|
|
- type: nauc_precision_at_1000_std
|
|
value: 34.896942379633316
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -11.425102107370272
|
|
- type: nauc_precision_at_100_max
|
|
value: 11.216164840931667
|
|
- type: nauc_precision_at_100_std
|
|
value: 27.722125456439343
|
|
- type: nauc_precision_at_10_diff1
|
|
value: -3.1401539776631653
|
|
- type: nauc_precision_at_10_max
|
|
value: 10.416214004945402
|
|
- type: nauc_precision_at_10_std
|
|
value: 10.251563605515335
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 71.30652188008001
|
|
- type: nauc_precision_at_1_max
|
|
value: 32.60802008342313
|
|
- type: nauc_precision_at_1_std
|
|
value: -12.29496015891874
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -6.456921653790667
|
|
- type: nauc_precision_at_20_max
|
|
value: 10.23022445081364
|
|
- type: nauc_precision_at_20_std
|
|
value: 15.935771905722302
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 8.38156786039047
|
|
- type: nauc_precision_at_3_max
|
|
value: 12.08129239567508
|
|
- type: nauc_precision_at_3_std
|
|
value: 0.05626041327325479
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 2.4102262974666653
|
|
- type: nauc_precision_at_5_max
|
|
value: 11.160384909564122
|
|
- type: nauc_precision_at_5_std
|
|
value: 4.587163311214582
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: -18.605360521019925
|
|
- type: nauc_recall_at_1000_max
|
|
value: 13.992651128348363
|
|
- type: nauc_recall_at_1000_std
|
|
value: 34.89694237963353
|
|
- type: nauc_recall_at_100_diff1
|
|
value: -11.425102107370193
|
|
- type: nauc_recall_at_100_max
|
|
value: 11.216164840931476
|
|
- type: nauc_recall_at_100_std
|
|
value: 27.72212545643919
|
|
- type: nauc_recall_at_10_diff1
|
|
value: -3.140153977663016
|
|
- type: nauc_recall_at_10_max
|
|
value: 10.416214004945413
|
|
- type: nauc_recall_at_10_std
|
|
value: 10.251563605515395
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 71.30652188008001
|
|
- type: nauc_recall_at_1_max
|
|
value: 32.60802008342313
|
|
- type: nauc_recall_at_1_std
|
|
value: -12.29496015891874
|
|
- type: nauc_recall_at_20_diff1
|
|
value: -6.45692165379055
|
|
- type: nauc_recall_at_20_max
|
|
value: 10.230224450813735
|
|
- type: nauc_recall_at_20_std
|
|
value: 15.935771905722335
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 8.381567860390362
|
|
- type: nauc_recall_at_3_max
|
|
value: 12.081292395675078
|
|
- type: nauc_recall_at_3_std
|
|
value: 0.05626041327321052
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 2.4102262974666355
|
|
- type: nauc_recall_at_5_max
|
|
value: 11.160384909564078
|
|
- type: nauc_recall_at_5_std
|
|
value: 4.587163311214529
|
|
- type: ndcg_at_1
|
|
value: 77.78500000000001
|
|
- type: ndcg_at_10
|
|
value: 68.65299999999999
|
|
- type: ndcg_at_100
|
|
value: 71.69200000000001
|
|
- type: ndcg_at_1000
|
|
value: 72.869
|
|
- type: ndcg_at_20
|
|
value: 70.078
|
|
- type: ndcg_at_3
|
|
value: 63.568000000000005
|
|
- type: ndcg_at_5
|
|
value: 66.402
|
|
- type: precision_at_1
|
|
value: 77.78500000000001
|
|
- type: precision_at_10
|
|
value: 14.386
|
|
- type: precision_at_100
|
|
value: 1.672
|
|
- type: precision_at_1000
|
|
value: 0.183
|
|
- type: precision_at_20
|
|
value: 7.6499999999999995
|
|
- type: precision_at_3
|
|
value: 40.473
|
|
- type: precision_at_5
|
|
value: 26.515
|
|
- type: recall_at_1
|
|
value: 38.893
|
|
- type: recall_at_10
|
|
value: 71.931
|
|
- type: recall_at_100
|
|
value: 83.619
|
|
- type: recall_at_1000
|
|
value: 91.431
|
|
- type: recall_at_20
|
|
value: 76.496
|
|
- type: recall_at_3
|
|
value: 60.709
|
|
- type: recall_at_5
|
|
value: 66.286
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB ImdbClassification
|
|
revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
|
|
split: test
|
|
type: mteb/imdb
|
|
metrics:
|
|
- type: accuracy
|
|
value: 95.0268
|
|
- type: ap
|
|
value: 92.72653250341486
|
|
- type: ap_weighted
|
|
value: 92.72653250341486
|
|
- type: f1
|
|
value: 95.02503365717179
|
|
- type: f1_weighted
|
|
value: 95.02503365717179
|
|
- type: main_score
|
|
value: 95.0268
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB MSMARCO
|
|
revision: c5a29a104738b98a9e76336939199e264163d4a0
|
|
split: dev
|
|
type: mteb/msmarco
|
|
metrics:
|
|
- type: main_score
|
|
value: 35.191
|
|
- type: map_at_1
|
|
value: 16.139
|
|
- type: map_at_10
|
|
value: 28.101
|
|
- type: map_at_100
|
|
value: 29.461
|
|
- type: map_at_1000
|
|
value: 29.515
|
|
- type: map_at_20
|
|
value: 28.936
|
|
- type: map_at_3
|
|
value: 23.954
|
|
- type: map_at_5
|
|
value: 26.308999999999997
|
|
- type: mrr_at_1
|
|
value: 16.59025787965616
|
|
- type: mrr_at_10
|
|
value: 28.583100241051344
|
|
- type: mrr_at_100
|
|
value: 29.89488944200741
|
|
- type: mrr_at_1000
|
|
value: 29.94198818201922
|
|
- type: mrr_at_20
|
|
value: 29.397153289126486
|
|
- type: mrr_at_3
|
|
value: 24.512893982807853
|
|
- type: mrr_at_5
|
|
value: 26.840974212034318
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 29.92308133337915
|
|
- type: nauc_map_at_1000_max
|
|
value: -4.792013160789208
|
|
- type: nauc_map_at_1000_std
|
|
value: -20.365722765519205
|
|
- type: nauc_map_at_100_diff1
|
|
value: 29.927608009586475
|
|
- type: nauc_map_at_100_max
|
|
value: -4.813011061550381
|
|
- type: nauc_map_at_100_std
|
|
value: -20.34066079647475
|
|
- type: nauc_map_at_10_diff1
|
|
value: 29.85964417677257
|
|
- type: nauc_map_at_10_max
|
|
value: -5.020819297438392
|
|
- type: nauc_map_at_10_std
|
|
value: -21.185600868900707
|
|
- type: nauc_map_at_1_diff1
|
|
value: 31.91727354325134
|
|
- type: nauc_map_at_1_max
|
|
value: -3.3836191178002637
|
|
- type: nauc_map_at_1_std
|
|
value: -18.94420033626203
|
|
- type: nauc_map_at_20_diff1
|
|
value: 29.909409775064265
|
|
- type: nauc_map_at_20_max
|
|
value: -4.882624170262229
|
|
- type: nauc_map_at_20_std
|
|
value: -20.737422787243176
|
|
- type: nauc_map_at_3_diff1
|
|
value: 29.96619551770926
|
|
- type: nauc_map_at_3_max
|
|
value: -4.521984358305567
|
|
- type: nauc_map_at_3_std
|
|
value: -20.675567430573214
|
|
- type: nauc_map_at_5_diff1
|
|
value: 29.672157845793336
|
|
- type: nauc_map_at_5_max
|
|
value: -4.784226867946108
|
|
- type: nauc_map_at_5_std
|
|
value: -21.090554010504313
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 29.57786251899136
|
|
- type: nauc_mrr_at_1000_max
|
|
value: -4.554864207268301
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -20.124071230468733
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 29.57869911178864
|
|
- type: nauc_mrr_at_100_max
|
|
value: -4.568738533954914
|
|
- type: nauc_mrr_at_100_std
|
|
value: -20.097461372571754
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 29.50101055760309
|
|
- type: nauc_mrr_at_10_max
|
|
value: -4.699465165716407
|
|
- type: nauc_mrr_at_10_std
|
|
value: -20.85880213075095
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 31.5283761916309
|
|
- type: nauc_mrr_at_1_max
|
|
value: -3.2410968598060226
|
|
- type: nauc_mrr_at_1_std
|
|
value: -18.877804738741848
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 29.55469091898283
|
|
- type: nauc_mrr_at_20_max
|
|
value: -4.6114669798589585
|
|
- type: nauc_mrr_at_20_std
|
|
value: -20.433076769992457
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 29.62441465248462
|
|
- type: nauc_mrr_at_3_max
|
|
value: -4.317634456438896
|
|
- type: nauc_mrr_at_3_std
|
|
value: -20.545356421989975
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 29.3174731757817
|
|
- type: nauc_mrr_at_5_max
|
|
value: -4.524554398532275
|
|
- type: nauc_mrr_at_5_std
|
|
value: -20.87564955466439
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 29.417049449756306
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: -4.429863573283831
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -18.672687178180762
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 29.52545788575206
|
|
- type: nauc_ndcg_at_100_max
|
|
value: -4.839548635918072
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -17.445902376477168
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 29.349337034114708
|
|
- type: nauc_ndcg_at_10_max
|
|
value: -5.654575625474153
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -21.867391862075433
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 31.5283761916309
|
|
- type: nauc_ndcg_at_1_max
|
|
value: -3.2410968598060226
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -18.877804738741848
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 29.478679665234736
|
|
- type: nauc_ndcg_at_20_max
|
|
value: -5.348280869926551
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -20.32251566103604
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 29.41586840338385
|
|
- type: nauc_ndcg_at_3_max
|
|
value: -4.737448759293484
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -21.114595209094198
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 28.95897834819025
|
|
- type: nauc_ndcg_at_5_max
|
|
value: -5.144033504465505
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -21.73482008242439
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -4.773246418887565
|
|
- type: nauc_precision_at_1000_max
|
|
value: 18.94086713593158
|
|
- type: nauc_precision_at_1000_std
|
|
value: 14.940921913943725
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 15.529104524208284
|
|
- type: nauc_precision_at_100_max
|
|
value: 4.152043132226839
|
|
- type: nauc_precision_at_100_std
|
|
value: 15.362588630598356
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 26.327252473718293
|
|
- type: nauc_precision_at_10_max
|
|
value: -6.385696358427295
|
|
- type: nauc_precision_at_10_std
|
|
value: -22.43695468265468
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 31.5283761916309
|
|
- type: nauc_precision_at_1_max
|
|
value: -3.2410968598060226
|
|
- type: nauc_precision_at_1_std
|
|
value: -18.877804738741848
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 25.09386904802987
|
|
- type: nauc_precision_at_20_max
|
|
value: -4.384006847324815
|
|
- type: nauc_precision_at_20_std
|
|
value: -15.476174306633775
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 27.88147581285313
|
|
- type: nauc_precision_at_3_max
|
|
value: -5.10330889992625
|
|
- type: nauc_precision_at_3_std
|
|
value: -22.17804890064486
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 26.673260429548385
|
|
- type: nauc_precision_at_5_max
|
|
value: -5.849985467654149
|
|
- type: nauc_precision_at_5_std
|
|
value: -23.22704929951935
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 11.078337058729081
|
|
- type: nauc_recall_at_1000_max
|
|
value: 29.31329518339392
|
|
- type: nauc_recall_at_1000_std
|
|
value: 61.689932707089845
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 27.694660226790095
|
|
- type: nauc_recall_at_100_max
|
|
value: -4.662880554456902
|
|
- type: nauc_recall_at_100_std
|
|
value: 17.291575712920476
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 28.14620642731046
|
|
- type: nauc_recall_at_10_max
|
|
value: -7.883918071832969
|
|
- type: nauc_recall_at_10_std
|
|
value: -23.85382911185965
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 31.91727354325134
|
|
- type: nauc_recall_at_1_max
|
|
value: -3.3836191178002637
|
|
- type: nauc_recall_at_1_std
|
|
value: -18.94420033626203
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 28.411188230736368
|
|
- type: nauc_recall_at_20_max
|
|
value: -7.489052404904147
|
|
- type: nauc_recall_at_20_std
|
|
value: -17.923010929300084
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 28.13888531840714
|
|
- type: nauc_recall_at_3_max
|
|
value: -5.385513963117635
|
|
- type: nauc_recall_at_3_std
|
|
value: -22.09635477229696
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 27.197531472369057
|
|
- type: nauc_recall_at_5_max
|
|
value: -6.204044942502606
|
|
- type: nauc_recall_at_5_std
|
|
value: -23.25902678179945
|
|
- type: ndcg_at_1
|
|
value: 16.59
|
|
- type: ndcg_at_10
|
|
value: 35.191
|
|
- type: ndcg_at_100
|
|
value: 41.778999999999996
|
|
- type: ndcg_at_1000
|
|
value: 43.126999999999995
|
|
- type: ndcg_at_20
|
|
value: 38.153
|
|
- type: ndcg_at_3
|
|
value: 26.718999999999998
|
|
- type: ndcg_at_5
|
|
value: 30.919999999999998
|
|
- type: precision_at_1
|
|
value: 16.59
|
|
- type: precision_at_10
|
|
value: 5.992999999999999
|
|
- type: precision_at_100
|
|
value: 0.927
|
|
- type: precision_at_1000
|
|
value: 0.104
|
|
- type: precision_at_20
|
|
value: 3.6020000000000003
|
|
- type: precision_at_3
|
|
value: 11.815000000000001
|
|
- type: precision_at_5
|
|
value: 9.218
|
|
- type: recall_at_1
|
|
value: 16.139
|
|
- type: recall_at_10
|
|
value: 57.272999999999996
|
|
- type: recall_at_100
|
|
value: 87.819
|
|
- type: recall_at_1000
|
|
value: 98.10900000000001
|
|
- type: recall_at_20
|
|
value: 68.77
|
|
- type: recall_at_3
|
|
value: 34.172999999999995
|
|
- type: recall_at_5
|
|
value: 44.259
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: en
|
|
name: MTEB MTOPDomainClassification (en)
|
|
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
|
|
split: test
|
|
type: mteb/mtop_domain
|
|
metrics:
|
|
- type: accuracy
|
|
value: 97.57637938896491
|
|
- type: f1
|
|
value: 97.39941554989736
|
|
- type: f1_weighted
|
|
value: 97.58495129362304
|
|
- type: main_score
|
|
value: 97.57637938896491
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: en
|
|
name: MTEB MTOPIntentClassification (en)
|
|
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
|
|
split: test
|
|
type: mteb/mtop_intent
|
|
metrics:
|
|
- type: accuracy
|
|
value: 86.08071135430917
|
|
- type: f1
|
|
value: 60.67695519910473
|
|
- type: f1_weighted
|
|
value: 86.22253292076088
|
|
- type: main_score
|
|
value: 86.08071135430917
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: en
|
|
name: MTEB MassiveIntentClassification (en)
|
|
revision: 4672e20407010da34463acc759c162ca9734bca6
|
|
split: test
|
|
type: mteb/amazon_massive_intent
|
|
metrics:
|
|
- type: accuracy
|
|
value: 74.9394754539341
|
|
- type: f1
|
|
value: 71.84595519829237
|
|
- type: f1_weighted
|
|
value: 73.7724380212837
|
|
- type: main_score
|
|
value: 74.9394754539341
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: en
|
|
name: MTEB MassiveScenarioClassification (en)
|
|
revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
|
|
split: test
|
|
type: mteb/amazon_massive_scenario
|
|
metrics:
|
|
- type: accuracy
|
|
value: 82.0611970410222
|
|
- type: f1
|
|
value: 80.96764019308867
|
|
- type: f1_weighted
|
|
value: 81.75048816703206
|
|
- type: main_score
|
|
value: 82.0611970410222
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB MedrxivClusteringP2P
|
|
revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
|
|
split: test
|
|
type: mteb/medrxiv-clustering-p2p
|
|
metrics:
|
|
- type: main_score
|
|
value: 35.535315182381275
|
|
- type: v_measure
|
|
value: 35.535315182381275
|
|
- type: v_measure_std
|
|
value: 1.2947784991789062
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB MedrxivClusteringS2S
|
|
revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
|
|
split: test
|
|
type: mteb/medrxiv-clustering-s2s
|
|
metrics:
|
|
- type: main_score
|
|
value: 32.701317380058356
|
|
- type: v_measure
|
|
value: 32.701317380058356
|
|
- type: v_measure_std
|
|
value: 1.212859415243672
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB MindSmallReranking
|
|
revision: 59042f120c80e8afa9cdbb224f67076cec0fc9a7
|
|
split: test
|
|
type: mteb/mind_small
|
|
metrics:
|
|
- type: main_score
|
|
value: 31.586146859630325
|
|
- type: map
|
|
value: 31.586146859630325
|
|
- type: mrr
|
|
value: 32.74920599119196
|
|
- type: nAUC_map_diff1
|
|
value: 11.669586995601716
|
|
- type: nAUC_map_max
|
|
value: -19.043343922416184
|
|
- type: nAUC_map_std
|
|
value: -0.002926267520007513
|
|
- type: nAUC_mrr_diff1
|
|
value: 11.132898797866952
|
|
- type: nAUC_mrr_max
|
|
value: -13.521554137760747
|
|
- type: nAUC_mrr_std
|
|
value: 1.6662256096686372
|
|
task:
|
|
type: Reranking
|
|
- dataset:
|
|
config: default
|
|
name: MTEB NFCorpus
|
|
revision: ec0fa4fe99da2ff19ca1214b7966684033a58814
|
|
split: test
|
|
type: mteb/nfcorpus
|
|
metrics:
|
|
- type: main_score
|
|
value: 34.493
|
|
- type: map_at_1
|
|
value: 5.469
|
|
- type: map_at_10
|
|
value: 12.681999999999999
|
|
- type: map_at_100
|
|
value: 16.136
|
|
- type: map_at_1000
|
|
value: 17.574
|
|
- type: map_at_20
|
|
value: 14.063
|
|
- type: map_at_3
|
|
value: 9.252
|
|
- type: map_at_5
|
|
value: 11.03
|
|
- type: mrr_at_1
|
|
value: 43.962848297213625
|
|
- type: mrr_at_10
|
|
value: 53.748095729519854
|
|
- type: mrr_at_100
|
|
value: 54.31371383821993
|
|
- type: mrr_at_1000
|
|
value: 54.34550446424
|
|
- type: mrr_at_20
|
|
value: 54.05753630252571
|
|
- type: mrr_at_3
|
|
value: 51.34158926728587
|
|
- type: mrr_at_5
|
|
value: 52.951496388028886
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 22.42945451651053
|
|
- type: nauc_map_at_1000_max
|
|
value: 25.044939905094555
|
|
- type: nauc_map_at_1000_std
|
|
value: 14.6947376252321
|
|
- type: nauc_map_at_100_diff1
|
|
value: 24.05126858377848
|
|
- type: nauc_map_at_100_max
|
|
value: 24.260286968462943
|
|
- type: nauc_map_at_100_std
|
|
value: 11.274560706750162
|
|
- type: nauc_map_at_10_diff1
|
|
value: 28.610449405636412
|
|
- type: nauc_map_at_10_max
|
|
value: 17.669350840567517
|
|
- type: nauc_map_at_10_std
|
|
value: -0.5603965547026133
|
|
- type: nauc_map_at_1_diff1
|
|
value: 44.546139576048574
|
|
- type: nauc_map_at_1_max
|
|
value: 3.5966098414779686
|
|
- type: nauc_map_at_1_std
|
|
value: -15.204463497276185
|
|
- type: nauc_map_at_20_diff1
|
|
value: 26.93971089998854
|
|
- type: nauc_map_at_20_max
|
|
value: 20.89952744553902
|
|
- type: nauc_map_at_20_std
|
|
value: 4.323667205452283
|
|
- type: nauc_map_at_3_diff1
|
|
value: 34.03753780494977
|
|
- type: nauc_map_at_3_max
|
|
value: 10.951970261908517
|
|
- type: nauc_map_at_3_std
|
|
value: -8.942935860299977
|
|
- type: nauc_map_at_5_diff1
|
|
value: 31.13647526539977
|
|
- type: nauc_map_at_5_max
|
|
value: 13.55486409562657
|
|
- type: nauc_map_at_5_std
|
|
value: -6.285335121924455
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 33.04380727929978
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 40.97460730083534
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 22.68307762886138
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 33.038505852668905
|
|
- type: nauc_mrr_at_100_max
|
|
value: 41.004813808229976
|
|
- type: nauc_mrr_at_100_std
|
|
value: 22.727078227914703
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 32.945102642427294
|
|
- type: nauc_mrr_at_10_max
|
|
value: 40.59087425732438
|
|
- type: nauc_mrr_at_10_std
|
|
value: 22.2969763977488
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 34.55355095202985
|
|
- type: nauc_mrr_at_1_max
|
|
value: 34.35691144716251
|
|
- type: nauc_mrr_at_1_std
|
|
value: 16.025738199559136
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 33.01684360381644
|
|
- type: nauc_mrr_at_20_max
|
|
value: 40.82433798731643
|
|
- type: nauc_mrr_at_20_std
|
|
value: 22.56838707992269
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 33.2000664328818
|
|
- type: nauc_mrr_at_3_max
|
|
value: 40.65557927809233
|
|
- type: nauc_mrr_at_3_std
|
|
value: 21.640445622194292
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 33.14724263980201
|
|
- type: nauc_mrr_at_5_max
|
|
value: 40.37502720649393
|
|
- type: nauc_mrr_at_5_std
|
|
value: 20.91483571628846
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 23.13999445390973
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 40.904356797688244
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 31.135131225973755
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 21.60764588276507
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 34.72455917031235
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 26.084570343364895
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 21.273666650824712
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 36.42637032684147
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 25.854371107614753
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 35.40190534464431
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 34.09394953710087
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 15.082336268368568
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 20.629683502494935
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 35.01440571472175
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 26.1516323412204
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 27.314585132007803
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 38.19301088947643
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 22.37292581921333
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 24.033794102904647
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 36.466778291326506
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 23.15763774408816
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -13.984096369493178
|
|
- type: nauc_precision_at_1000_max
|
|
value: 8.50221544384146
|
|
- type: nauc_precision_at_1000_std
|
|
value: 35.62592696752026
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -12.115042643624523
|
|
- type: nauc_precision_at_100_max
|
|
value: 21.139964351279062
|
|
- type: nauc_precision_at_100_std
|
|
value: 45.41323150126541
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 3.5604358960435594
|
|
- type: nauc_precision_at_10_max
|
|
value: 38.21371536948471
|
|
- type: nauc_precision_at_10_std
|
|
value: 40.093467246870674
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 34.55355095202985
|
|
- type: nauc_precision_at_1_max
|
|
value: 34.35691144716251
|
|
- type: nauc_precision_at_1_std
|
|
value: 16.025738199559136
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -2.2994929672216142
|
|
- type: nauc_precision_at_20_max
|
|
value: 33.41182551515417
|
|
- type: nauc_precision_at_20_std
|
|
value: 42.926074063475376
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 17.026846985190286
|
|
- type: nauc_precision_at_3_max
|
|
value: 40.78926087324481
|
|
- type: nauc_precision_at_3_std
|
|
value: 28.26154405706766
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 10.066105504177528
|
|
- type: nauc_precision_at_5_max
|
|
value: 38.397299240351515
|
|
- type: nauc_precision_at_5_std
|
|
value: 31.504726528569105
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 5.433767085525343
|
|
- type: nauc_recall_at_1000_max
|
|
value: 17.082294989371675
|
|
- type: nauc_recall_at_1000_std
|
|
value: 17.867147762696924
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 10.513494371628159
|
|
- type: nauc_recall_at_100_max
|
|
value: 19.63867418942476
|
|
- type: nauc_recall_at_100_std
|
|
value: 14.421450754520809
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 22.750728383486376
|
|
- type: nauc_recall_at_10_max
|
|
value: 15.735611146890621
|
|
- type: nauc_recall_at_10_std
|
|
value: -0.40290229377136233
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 44.546139576048574
|
|
- type: nauc_recall_at_1_max
|
|
value: 3.5966098414779686
|
|
- type: nauc_recall_at_1_std
|
|
value: -15.204463497276185
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 22.44097500377964
|
|
- type: nauc_recall_at_20_max
|
|
value: 19.99783526750806
|
|
- type: nauc_recall_at_20_std
|
|
value: 5.831968175648315
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 30.742501145388644
|
|
- type: nauc_recall_at_3_max
|
|
value: 11.887713348765457
|
|
- type: nauc_recall_at_3_std
|
|
value: -7.507756416467706
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 25.251057623903268
|
|
- type: nauc_recall_at_5_max
|
|
value: 11.530971742020508
|
|
- type: nauc_recall_at_5_std
|
|
value: -6.9727238554804005
|
|
- type: ndcg_at_1
|
|
value: 42.57
|
|
- type: ndcg_at_10
|
|
value: 34.493
|
|
- type: ndcg_at_100
|
|
value: 31.912000000000003
|
|
- type: ndcg_at_1000
|
|
value: 40.485
|
|
- type: ndcg_at_20
|
|
value: 32.314
|
|
- type: ndcg_at_3
|
|
value: 39.546
|
|
- type: ndcg_at_5
|
|
value: 38.009
|
|
- type: precision_at_1
|
|
value: 43.963
|
|
- type: precision_at_10
|
|
value: 25.728
|
|
- type: precision_at_100
|
|
value: 8.297
|
|
- type: precision_at_1000
|
|
value: 2.094
|
|
- type: precision_at_20
|
|
value: 19.288
|
|
- type: precision_at_3
|
|
value: 37.564
|
|
- type: precision_at_5
|
|
value: 33.375
|
|
- type: recall_at_1
|
|
value: 5.469
|
|
- type: recall_at_10
|
|
value: 16.733
|
|
- type: recall_at_100
|
|
value: 32.867000000000004
|
|
- type: recall_at_1000
|
|
value: 63.873000000000005
|
|
- type: recall_at_20
|
|
value: 20.312
|
|
- type: recall_at_3
|
|
value: 10.386
|
|
- type: recall_at_5
|
|
value: 13.679
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB NQ
|
|
revision: b774495ed302d8c44a3a7ea25c90dbce03968f31
|
|
split: test
|
|
type: mteb/nq
|
|
metrics:
|
|
- type: main_score
|
|
value: 49.539
|
|
- type: map_at_1
|
|
value: 26.016000000000002
|
|
- type: map_at_10
|
|
value: 41.23
|
|
- type: map_at_100
|
|
value: 42.466
|
|
- type: map_at_1000
|
|
value: 42.494
|
|
- type: map_at_20
|
|
value: 42.049
|
|
- type: map_at_3
|
|
value: 36.272999999999996
|
|
- type: map_at_5
|
|
value: 39.172000000000004
|
|
- type: mrr_at_1
|
|
value: 29.634994206257243
|
|
- type: mrr_at_10
|
|
value: 43.814949695598514
|
|
- type: mrr_at_100
|
|
value: 44.75158330890793
|
|
- type: mrr_at_1000
|
|
value: 44.76933611785972
|
|
- type: mrr_at_20
|
|
value: 44.450136580422104
|
|
- type: mrr_at_3
|
|
value: 39.56160679799143
|
|
- type: mrr_at_5
|
|
value: 42.083333333333336
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 31.377733390159623
|
|
- type: nauc_map_at_1000_max
|
|
value: 10.852802240297759
|
|
- type: nauc_map_at_1000_std
|
|
value: -8.156368414989963
|
|
- type: nauc_map_at_100_diff1
|
|
value: 31.37926107010834
|
|
- type: nauc_map_at_100_max
|
|
value: 10.866567017386616
|
|
- type: nauc_map_at_100_std
|
|
value: -8.13083658675661
|
|
- type: nauc_map_at_10_diff1
|
|
value: 31.302395420970413
|
|
- type: nauc_map_at_10_max
|
|
value: 10.696471249499485
|
|
- type: nauc_map_at_10_std
|
|
value: -8.608828614048587
|
|
- type: nauc_map_at_1_diff1
|
|
value: 34.515378947817545
|
|
- type: nauc_map_at_1_max
|
|
value: 8.23278785130009
|
|
- type: nauc_map_at_1_std
|
|
value: -8.790135666737623
|
|
- type: nauc_map_at_20_diff1
|
|
value: 31.405784027747636
|
|
- type: nauc_map_at_20_max
|
|
value: 10.743222784357599
|
|
- type: nauc_map_at_20_std
|
|
value: -8.336520716356294
|
|
- type: nauc_map_at_3_diff1
|
|
value: 30.790756885918242
|
|
- type: nauc_map_at_3_max
|
|
value: 9.611996527156451
|
|
- type: nauc_map_at_3_std
|
|
value: -10.30419579409286
|
|
- type: nauc_map_at_5_diff1
|
|
value: 31.018701056437692
|
|
- type: nauc_map_at_5_max
|
|
value: 10.415471498676181
|
|
- type: nauc_map_at_5_std
|
|
value: -9.267868426130615
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 30.954103753005274
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 11.65610034595576
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -6.236607914879178
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 30.95419964742793
|
|
- type: nauc_mrr_at_100_max
|
|
value: 11.67074501272962
|
|
- type: nauc_mrr_at_100_std
|
|
value: -6.2148004414091504
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 30.909871849241917
|
|
- type: nauc_mrr_at_10_max
|
|
value: 11.663150347843652
|
|
- type: nauc_mrr_at_10_std
|
|
value: -6.412145873320221
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 33.69803436461973
|
|
- type: nauc_mrr_at_1_max
|
|
value: 9.810616582626253
|
|
- type: nauc_mrr_at_1_std
|
|
value: -6.5168183653335845
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 30.97036659208301
|
|
- type: nauc_mrr_at_20_max
|
|
value: 11.615291040042264
|
|
- type: nauc_mrr_at_20_std
|
|
value: -6.317206649176624
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 30.347687412668307
|
|
- type: nauc_mrr_at_3_max
|
|
value: 11.045997984562728
|
|
- type: nauc_mrr_at_3_std
|
|
value: -7.344237528386735
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 30.607591550974323
|
|
- type: nauc_mrr_at_5_max
|
|
value: 11.478687020349025
|
|
- type: nauc_mrr_at_5_std
|
|
value: -6.773130489910162
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 30.721715941822435
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 12.363613568822352
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -6.083916245339269
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 30.608831858292408
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 12.894646588979683
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -5.148801091143074
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 30.483771661792847
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 12.18129035771911
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -7.165744970217042
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 33.79845141868468
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 9.88864563426806
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -6.43552016535101
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 30.77504113488907
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 12.28245448589153
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -6.325276590452571
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 29.602918057743278
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 10.39055264754259
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -10.014843769784985
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 29.94463296702168
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 11.551920125900473
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -8.48593988495145
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -5.690546724212895
|
|
- type: nauc_precision_at_1000_max
|
|
value: 9.109366247129207
|
|
- type: nauc_precision_at_1000_std
|
|
value: 14.65465630262207
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -1.2336613199255233
|
|
- type: nauc_precision_at_100_max
|
|
value: 14.632255993612098
|
|
- type: nauc_precision_at_100_std
|
|
value: 20.106751006299508
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 16.156638161044377
|
|
- type: nauc_precision_at_10_max
|
|
value: 15.461271728023455
|
|
- type: nauc_precision_at_10_std
|
|
value: 4.613330902566019
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 33.79845141868468
|
|
- type: nauc_precision_at_1_max
|
|
value: 9.88864563426806
|
|
- type: nauc_precision_at_1_std
|
|
value: -6.43552016535101
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 10.833258836740004
|
|
- type: nauc_precision_at_20_max
|
|
value: 14.399547246551503
|
|
- type: nauc_precision_at_20_std
|
|
value: 10.691750912308304
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 23.440967729505452
|
|
- type: nauc_precision_at_3_max
|
|
value: 12.708378101618688
|
|
- type: nauc_precision_at_3_std
|
|
value: -7.2002199170375105
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 20.632161061662867
|
|
- type: nauc_precision_at_5_max
|
|
value: 14.803138265646187
|
|
- type: nauc_precision_at_5_std
|
|
value: -1.9170585171231866
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 17.469814268756277
|
|
- type: nauc_recall_at_1000_max
|
|
value: 67.91132861575576
|
|
- type: nauc_recall_at_1000_std
|
|
value: 59.719785001643054
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 20.871489158949146
|
|
- type: nauc_recall_at_100_max
|
|
value: 42.25616221901811
|
|
- type: nauc_recall_at_100_std
|
|
value: 41.83257983711543
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 26.116159187824273
|
|
- type: nauc_recall_at_10_max
|
|
value: 15.673928195577544
|
|
- type: nauc_recall_at_10_std
|
|
value: -4.068034337550412
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 34.515378947817545
|
|
- type: nauc_recall_at_1_max
|
|
value: 8.23278785130009
|
|
- type: nauc_recall_at_1_std
|
|
value: -8.790135666737623
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 26.830515495608314
|
|
- type: nauc_recall_at_20_max
|
|
value: 17.956121895077352
|
|
- type: nauc_recall_at_20_std
|
|
value: 1.8149755315374414
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 25.57777694351554
|
|
- type: nauc_recall_at_3_max
|
|
value: 10.768605841163243
|
|
- type: nauc_recall_at_3_std
|
|
value: -11.548054988544685
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 25.69071002325843
|
|
- type: nauc_recall_at_5_max
|
|
value: 13.248151375739594
|
|
- type: nauc_recall_at_5_std
|
|
value: -8.31127808515032
|
|
- type: ndcg_at_1
|
|
value: 29.605999999999998
|
|
- type: ndcg_at_10
|
|
value: 49.539
|
|
- type: ndcg_at_100
|
|
value: 54.67999999999999
|
|
- type: ndcg_at_1000
|
|
value: 55.287
|
|
- type: ndcg_at_20
|
|
value: 52.196
|
|
- type: ndcg_at_3
|
|
value: 40.111999999999995
|
|
- type: ndcg_at_5
|
|
value: 44.983000000000004
|
|
- type: precision_at_1
|
|
value: 29.605999999999998
|
|
- type: precision_at_10
|
|
value: 8.607
|
|
- type: precision_at_100
|
|
value: 1.147
|
|
- type: precision_at_1000
|
|
value: 0.121
|
|
- type: precision_at_20
|
|
value: 4.938
|
|
- type: precision_at_3
|
|
value: 18.627
|
|
- type: precision_at_5
|
|
value: 13.927999999999999
|
|
- type: recall_at_1
|
|
value: 26.016000000000002
|
|
- type: recall_at_10
|
|
value: 72.51100000000001
|
|
- type: recall_at_100
|
|
value: 94.60499999999999
|
|
- type: recall_at_1000
|
|
value: 99.054
|
|
- type: recall_at_20
|
|
value: 82.353
|
|
- type: recall_at_3
|
|
value: 47.989
|
|
- type: recall_at_5
|
|
value: 59.243
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB QuoraRetrieval
|
|
revision: e4e08e0b7dbe3c8700f0daef558ff32256715259
|
|
split: test
|
|
type: mteb/quora
|
|
metrics:
|
|
- type: main_score
|
|
value: 89.387
|
|
- type: map_at_1
|
|
value: 71.61699999999999
|
|
- type: map_at_10
|
|
value: 85.785
|
|
- type: map_at_100
|
|
value: 86.407
|
|
- type: map_at_1000
|
|
value: 86.42
|
|
- type: map_at_20
|
|
value: 86.206
|
|
- type: map_at_3
|
|
value: 82.867
|
|
- type: map_at_5
|
|
value: 84.736
|
|
- type: mrr_at_1
|
|
value: 82.49
|
|
- type: mrr_at_10
|
|
value: 88.59147619047603
|
|
- type: mrr_at_100
|
|
value: 88.67100295673903
|
|
- type: mrr_at_1000
|
|
value: 88.67132516200078
|
|
- type: mrr_at_20
|
|
value: 88.6561804240649
|
|
- type: mrr_at_3
|
|
value: 87.72499999999982
|
|
- type: mrr_at_5
|
|
value: 88.34599999999975
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 77.75322227698767
|
|
- type: nauc_map_at_1000_max
|
|
value: 27.15325474904755
|
|
- type: nauc_map_at_1000_std
|
|
value: -45.950703261401266
|
|
- type: nauc_map_at_100_diff1
|
|
value: 77.75046471198675
|
|
- type: nauc_map_at_100_max
|
|
value: 27.125684918574887
|
|
- type: nauc_map_at_100_std
|
|
value: -46.00793046653974
|
|
- type: nauc_map_at_10_diff1
|
|
value: 77.96301805869726
|
|
- type: nauc_map_at_10_max
|
|
value: 26.63787475984541
|
|
- type: nauc_map_at_10_std
|
|
value: -48.2092244990593
|
|
- type: nauc_map_at_1_diff1
|
|
value: 81.04847175933422
|
|
- type: nauc_map_at_1_max
|
|
value: 20.828021860691376
|
|
- type: nauc_map_at_1_std
|
|
value: -40.4427741623345
|
|
- type: nauc_map_at_20_diff1
|
|
value: 77.82691021180123
|
|
- type: nauc_map_at_20_max
|
|
value: 26.979439675350086
|
|
- type: nauc_map_at_20_std
|
|
value: -46.94206477224242
|
|
- type: nauc_map_at_3_diff1
|
|
value: 78.57251235300281
|
|
- type: nauc_map_at_3_max
|
|
value: 24.306776325229592
|
|
- type: nauc_map_at_3_std
|
|
value: -50.446232609379706
|
|
- type: nauc_map_at_5_diff1
|
|
value: 78.23538738312993
|
|
- type: nauc_map_at_5_max
|
|
value: 26.005150155221003
|
|
- type: nauc_map_at_5_std
|
|
value: -49.72081450369548
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 78.29655431237718
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 29.392496550114718
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -41.08607589889516
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 78.29662146607758
|
|
- type: nauc_mrr_at_100_max
|
|
value: 29.393300424020218
|
|
- type: nauc_mrr_at_100_std
|
|
value: -41.086465937239026
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 78.30206302797494
|
|
- type: nauc_mrr_at_10_max
|
|
value: 29.367617601691403
|
|
- type: nauc_mrr_at_10_std
|
|
value: -41.241804159667225
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 79.00375724290345
|
|
- type: nauc_mrr_at_1_max
|
|
value: 29.763227602149133
|
|
- type: nauc_mrr_at_1_std
|
|
value: -37.58361433096388
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 78.29875275029173
|
|
- type: nauc_mrr_at_20_max
|
|
value: 29.39463895371502
|
|
- type: nauc_mrr_at_20_std
|
|
value: -41.13808938179999
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 78.04981713424701
|
|
- type: nauc_mrr_at_3_max
|
|
value: 28.760448174610858
|
|
- type: nauc_mrr_at_3_std
|
|
value: -42.25770370267669
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 78.24030781659526
|
|
- type: nauc_mrr_at_5_max
|
|
value: 29.4627965404159
|
|
- type: nauc_mrr_at_5_std
|
|
value: -41.48382971161236
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 77.63586978346414
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 28.36041361858413
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -43.84956631664592
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 77.5782899412669
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 28.175349147299023
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -44.03384730985532
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 77.65612732311726
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 27.447934213310145
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -47.477846933136206
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 79.00375724290345
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 29.763227602149133
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -37.58361433096388
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 77.6857905925127
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 27.85965135690326
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -46.035623659567534
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 77.20000663124452
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 25.83926946771269
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -48.46047480037077
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 77.47304156996891
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 27.277217473255703
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -48.29036456924513
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -44.34289619168728
|
|
- type: nauc_precision_at_1000_max
|
|
value: -3.3267888861609882
|
|
- type: nauc_precision_at_1000_std
|
|
value: 40.7640626789122
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -44.40180123691582
|
|
- type: nauc_precision_at_100_max
|
|
value: -4.036815279824888
|
|
- type: nauc_precision_at_100_std
|
|
value: 40.258738157948144
|
|
- type: nauc_precision_at_10_diff1
|
|
value: -40.174969736392725
|
|
- type: nauc_precision_at_10_max
|
|
value: -1.2107921107014503
|
|
- type: nauc_precision_at_10_std
|
|
value: 26.914317558152383
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 79.00375724290345
|
|
- type: nauc_precision_at_1_max
|
|
value: 29.763227602149133
|
|
- type: nauc_precision_at_1_std
|
|
value: -37.58361433096388
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -42.997551532370395
|
|
- type: nauc_precision_at_20_max
|
|
value: -2.7260912846581435
|
|
- type: nauc_precision_at_20_std
|
|
value: 33.47494527610656
|
|
- type: nauc_precision_at_3_diff1
|
|
value: -21.172181060238913
|
|
- type: nauc_precision_at_3_max
|
|
value: 4.5591660958836835
|
|
- type: nauc_precision_at_3_std
|
|
value: 4.474651862429931
|
|
- type: nauc_precision_at_5_diff1
|
|
value: -33.376618015297154
|
|
- type: nauc_precision_at_5_max
|
|
value: 1.7302644290575764
|
|
- type: nauc_precision_at_5_std
|
|
value: 16.980633045220895
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 58.24743045343488
|
|
- type: nauc_recall_at_1000_max
|
|
value: -21.258859048904625
|
|
- type: nauc_recall_at_1000_std
|
|
value: 5.841590725271873
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 64.62432244425025
|
|
- type: nauc_recall_at_100_max
|
|
value: 11.438889005688548
|
|
- type: nauc_recall_at_100_std
|
|
value: -48.21565456849923
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 73.84516212868728
|
|
- type: nauc_recall_at_10_max
|
|
value: 21.581336143130912
|
|
- type: nauc_recall_at_10_std
|
|
value: -71.40446430175044
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 81.04847175933422
|
|
- type: nauc_recall_at_1_max
|
|
value: 20.828021860691376
|
|
- type: nauc_recall_at_1_std
|
|
value: -40.4427741623345
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 74.07490425440125
|
|
- type: nauc_recall_at_20_max
|
|
value: 22.741699258253938
|
|
- type: nauc_recall_at_20_std
|
|
value: -75.22910750948694
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 74.81258758793922
|
|
- type: nauc_recall_at_3_max
|
|
value: 19.256464797371688
|
|
- type: nauc_recall_at_3_std
|
|
value: -61.27309744783545
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 73.49570838483187
|
|
- type: nauc_recall_at_5_max
|
|
value: 22.485129670655922
|
|
- type: nauc_recall_at_5_std
|
|
value: -64.95541946081566
|
|
- type: ndcg_at_1
|
|
value: 82.49
|
|
- type: ndcg_at_10
|
|
value: 89.387
|
|
- type: ndcg_at_100
|
|
value: 90.464
|
|
- type: ndcg_at_1000
|
|
value: 90.533
|
|
- type: ndcg_at_20
|
|
value: 90.01599999999999
|
|
- type: ndcg_at_3
|
|
value: 86.726
|
|
- type: ndcg_at_5
|
|
value: 88.249
|
|
- type: precision_at_1
|
|
value: 82.49
|
|
- type: precision_at_10
|
|
value: 13.543
|
|
- type: precision_at_100
|
|
value: 1.5350000000000001
|
|
- type: precision_at_1000
|
|
value: 0.157
|
|
- type: precision_at_20
|
|
value: 7.185
|
|
- type: precision_at_3
|
|
value: 37.983
|
|
- type: precision_at_5
|
|
value: 24.954
|
|
- type: recall_at_1
|
|
value: 71.61699999999999
|
|
- type: recall_at_10
|
|
value: 96.207
|
|
- type: recall_at_100
|
|
value: 99.726
|
|
- type: recall_at_1000
|
|
value: 99.991
|
|
- type: recall_at_20
|
|
value: 98.188
|
|
- type: recall_at_3
|
|
value: 88.466
|
|
- type: recall_at_5
|
|
value: 92.83200000000001
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB RedditClustering
|
|
revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
|
|
split: test
|
|
type: mteb/reddit-clustering
|
|
metrics:
|
|
- type: main_score
|
|
value: 55.165421795067815
|
|
- type: v_measure
|
|
value: 55.165421795067815
|
|
- type: v_measure_std
|
|
value: 4.407201142010862
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB RedditClusteringP2P
|
|
revision: 385e3cb46b4cfa89021f56c4380204149d0efe33
|
|
split: test
|
|
type: mteb/reddit-clustering-p2p
|
|
metrics:
|
|
- type: main_score
|
|
value: 64.40104113698271
|
|
- type: v_measure
|
|
value: 64.40104113698271
|
|
- type: v_measure_std
|
|
value: 13.302523246335362
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SCIDOCS
|
|
revision: f8c2fcf00f625baaa80f62ec5bd9e1fff3b8ae88
|
|
split: test
|
|
type: mteb/scidocs
|
|
metrics:
|
|
- type: main_score
|
|
value: 20.429
|
|
- type: map_at_1
|
|
value: 4.868
|
|
- type: map_at_10
|
|
value: 12.27
|
|
- type: map_at_100
|
|
value: 14.332
|
|
- type: map_at_1000
|
|
value: 14.625
|
|
- type: map_at_20
|
|
value: 13.333
|
|
- type: map_at_3
|
|
value: 8.795
|
|
- type: map_at_5
|
|
value: 10.392
|
|
- type: mrr_at_1
|
|
value: 24.0
|
|
- type: mrr_at_10
|
|
value: 34.65333333333329
|
|
- type: mrr_at_100
|
|
value: 35.674251079833766
|
|
- type: mrr_at_1000
|
|
value: 35.73520785942911
|
|
- type: mrr_at_20
|
|
value: 35.22774876654128
|
|
- type: mrr_at_3
|
|
value: 31.166666666666664
|
|
- type: mrr_at_5
|
|
value: 33.281666666666624
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 17.399043123319522
|
|
- type: nauc_map_at_1000_max
|
|
value: 31.2734183775543
|
|
- type: nauc_map_at_1000_std
|
|
value: 17.077403711100832
|
|
- type: nauc_map_at_100_diff1
|
|
value: 17.403713887640865
|
|
- type: nauc_map_at_100_max
|
|
value: 31.27377201272501
|
|
- type: nauc_map_at_100_std
|
|
value: 16.87360366282937
|
|
- type: nauc_map_at_10_diff1
|
|
value: 17.359001538120168
|
|
- type: nauc_map_at_10_max
|
|
value: 30.468920168811948
|
|
- type: nauc_map_at_10_std
|
|
value: 13.380268231544715
|
|
- type: nauc_map_at_1_diff1
|
|
value: 21.421764472532455
|
|
- type: nauc_map_at_1_max
|
|
value: 22.406495947870948
|
|
- type: nauc_map_at_1_std
|
|
value: 7.278461750059741
|
|
- type: nauc_map_at_20_diff1
|
|
value: 17.309681501618616
|
|
- type: nauc_map_at_20_max
|
|
value: 30.723309484933736
|
|
- type: nauc_map_at_20_std
|
|
value: 15.103661234366466
|
|
- type: nauc_map_at_3_diff1
|
|
value: 19.21373088647576
|
|
- type: nauc_map_at_3_max
|
|
value: 28.20473469906757
|
|
- type: nauc_map_at_3_std
|
|
value: 8.112728025403056
|
|
- type: nauc_map_at_5_diff1
|
|
value: 18.058060387271972
|
|
- type: nauc_map_at_5_max
|
|
value: 30.126841947570814
|
|
- type: nauc_map_at_5_std
|
|
value: 10.52754125285907
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 19.441702934302622
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 25.596393086654306
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 12.03335655261492
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 19.45550504725835
|
|
- type: nauc_mrr_at_100_max
|
|
value: 25.616075945406113
|
|
- type: nauc_mrr_at_100_std
|
|
value: 12.064272002353919
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 19.439283557585867
|
|
- type: nauc_mrr_at_10_max
|
|
value: 25.630347604493288
|
|
- type: nauc_mrr_at_10_std
|
|
value: 12.031032042077703
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 21.522585669781943
|
|
- type: nauc_mrr_at_1_max
|
|
value: 22.47948118859334
|
|
- type: nauc_mrr_at_1_std
|
|
value: 7.382278936017263
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 19.41398208318509
|
|
- type: nauc_mrr_at_20_max
|
|
value: 25.627882587061446
|
|
- type: nauc_mrr_at_20_std
|
|
value: 12.073194157092846
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 19.605200019472257
|
|
- type: nauc_mrr_at_3_max
|
|
value: 25.325244620209876
|
|
- type: nauc_mrr_at_3_std
|
|
value: 9.621890524197736
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 19.39540169944071
|
|
- type: nauc_mrr_at_5_max
|
|
value: 25.603584740156034
|
|
- type: nauc_mrr_at_5_std
|
|
value: 11.176904475558963
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 16.677472512130397
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 30.803531883263386
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 24.271183062150264
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 17.36630862763037
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 31.94802140143363
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 23.50492571448407
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 16.96591943739385
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 29.983229462186355
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 16.195748077489096
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 21.522585669781943
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 22.47948118859334
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 7.382278936017263
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 16.95752397256498
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 30.17083071239411
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 18.58280825082001
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 18.84612108439313
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 27.98191818651593
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 9.424277024329921
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 17.508065912086675
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 29.611412732203608
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 12.623793734445126
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 6.265199779097322
|
|
- type: nauc_precision_at_1000_max
|
|
value: 20.008066463216657
|
|
- type: nauc_precision_at_1000_std
|
|
value: 35.98021866405677
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 11.877723135952802
|
|
- type: nauc_precision_at_100_max
|
|
value: 28.979530033834557
|
|
- type: nauc_precision_at_100_std
|
|
value: 33.61448120665875
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 13.347374773447774
|
|
- type: nauc_precision_at_10_max
|
|
value: 29.532781336663056
|
|
- type: nauc_precision_at_10_std
|
|
value: 20.58195880074721
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 21.522585669781943
|
|
- type: nauc_precision_at_1_max
|
|
value: 22.47948118859334
|
|
- type: nauc_precision_at_1_std
|
|
value: 7.382278936017263
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 12.623490622184555
|
|
- type: nauc_precision_at_20_max
|
|
value: 27.985132320790147
|
|
- type: nauc_precision_at_20_std
|
|
value: 24.017624920206707
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 17.586564287642346
|
|
- type: nauc_precision_at_3_max
|
|
value: 30.03148650786217
|
|
- type: nauc_precision_at_3_std
|
|
value: 10.379451374554094
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 14.824891223085926
|
|
- type: nauc_precision_at_5_max
|
|
value: 31.410239486293527
|
|
- type: nauc_precision_at_5_std
|
|
value: 15.624402346760954
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 6.310837044332995
|
|
- type: nauc_recall_at_1000_max
|
|
value: 20.095529403256776
|
|
- type: nauc_recall_at_1000_std
|
|
value: 36.54872612878018
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 12.038563848928966
|
|
- type: nauc_recall_at_100_max
|
|
value: 28.986817020127525
|
|
- type: nauc_recall_at_100_std
|
|
value: 33.54721716249713
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 13.26933896316366
|
|
- type: nauc_recall_at_10_max
|
|
value: 29.38186602785486
|
|
- type: nauc_recall_at_10_std
|
|
value: 20.275621953504526
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 21.421764472532455
|
|
- type: nauc_recall_at_1_max
|
|
value: 22.406495947870948
|
|
- type: nauc_recall_at_1_std
|
|
value: 7.278461750059741
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 12.570312459960123
|
|
- type: nauc_recall_at_20_max
|
|
value: 27.709620758158497
|
|
- type: nauc_recall_at_20_std
|
|
value: 23.607200666051515
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 17.403838471827413
|
|
- type: nauc_recall_at_3_max
|
|
value: 30.03567479942994
|
|
- type: nauc_recall_at_3_std
|
|
value: 10.168877039526405
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 14.617283448905278
|
|
- type: nauc_recall_at_5_max
|
|
value: 31.260794318671316
|
|
- type: nauc_recall_at_5_std
|
|
value: 15.292480271424239
|
|
- type: ndcg_at_1
|
|
value: 24.0
|
|
- type: ndcg_at_10
|
|
value: 20.429
|
|
- type: ndcg_at_100
|
|
value: 28.327999999999996
|
|
- type: ndcg_at_1000
|
|
value: 33.489999999999995
|
|
- type: ndcg_at_20
|
|
value: 23.236
|
|
- type: ndcg_at_3
|
|
value: 19.36
|
|
- type: ndcg_at_5
|
|
value: 16.866
|
|
- type: precision_at_1
|
|
value: 24.0
|
|
- type: precision_at_10
|
|
value: 10.58
|
|
- type: precision_at_100
|
|
value: 2.196
|
|
- type: precision_at_1000
|
|
value: 0.344
|
|
- type: precision_at_20
|
|
value: 6.9
|
|
- type: precision_at_3
|
|
value: 17.967
|
|
- type: precision_at_5
|
|
value: 14.74
|
|
- type: recall_at_1
|
|
value: 4.868
|
|
- type: recall_at_10
|
|
value: 21.47
|
|
- type: recall_at_100
|
|
value: 44.622
|
|
- type: recall_at_1000
|
|
value: 69.777
|
|
- type: recall_at_20
|
|
value: 28.028
|
|
- type: recall_at_3
|
|
value: 10.933
|
|
- type: recall_at_5
|
|
value: 14.948
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SICK-R
|
|
revision: 20a6d6f312dd54037fe07a32d58e5e168867909d
|
|
split: test
|
|
type: mteb/sickr-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 83.56937382314794
|
|
- type: cosine_spearman
|
|
value: 79.63245426461405
|
|
- type: euclidean_pearson
|
|
value: 81.23038281326936
|
|
- type: euclidean_spearman
|
|
value: 79.63246287500021
|
|
- type: main_score
|
|
value: 79.63245426461405
|
|
- type: manhattan_pearson
|
|
value: 81.22715334724163
|
|
- type: manhattan_spearman
|
|
value: 79.47235517811446
|
|
- type: pearson
|
|
value: 83.56937382314794
|
|
- type: spearman
|
|
value: 79.63245426461405
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB STS12
|
|
revision: a0d554a64d88156834ff5ae9920b964011b16384
|
|
split: test
|
|
type: mteb/sts12-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 87.94074172378106
|
|
- type: cosine_spearman
|
|
value: 81.49535893255212
|
|
- type: euclidean_pearson
|
|
value: 85.67127466141365
|
|
- type: euclidean_spearman
|
|
value: 81.49519105826656
|
|
- type: main_score
|
|
value: 81.49535893255212
|
|
- type: manhattan_pearson
|
|
value: 85.7939378777207
|
|
- type: manhattan_spearman
|
|
value: 81.68788285150019
|
|
- type: pearson
|
|
value: 87.94074172378106
|
|
- type: spearman
|
|
value: 81.49535893255212
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB STS13
|
|
revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
|
|
split: test
|
|
type: mteb/sts13-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 83.13868249088958
|
|
- type: cosine_spearman
|
|
value: 84.49255715794354
|
|
- type: euclidean_pearson
|
|
value: 83.94702761019037
|
|
- type: euclidean_spearman
|
|
value: 84.49261181536836
|
|
- type: main_score
|
|
value: 84.49255715794354
|
|
- type: manhattan_pearson
|
|
value: 84.05461037469608
|
|
- type: manhattan_spearman
|
|
value: 84.58504951653568
|
|
- type: pearson
|
|
value: 83.13868249088958
|
|
- type: spearman
|
|
value: 84.49255715794354
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB STS14
|
|
revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
|
|
split: test
|
|
type: mteb/sts14-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 80.86639951141099
|
|
- type: cosine_spearman
|
|
value: 80.05601661201852
|
|
- type: euclidean_pearson
|
|
value: 80.97495767233256
|
|
- type: euclidean_spearman
|
|
value: 80.05600716279979
|
|
- type: main_score
|
|
value: 80.05601661201852
|
|
- type: manhattan_pearson
|
|
value: 80.68673997093622
|
|
- type: manhattan_spearman
|
|
value: 79.895855702411
|
|
- type: pearson
|
|
value: 80.86639951141099
|
|
- type: spearman
|
|
value: 80.05601661201852
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB STS15
|
|
revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
|
|
split: test
|
|
type: mteb/sts15-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 84.13791770600066
|
|
- type: cosine_spearman
|
|
value: 86.54345663501209
|
|
- type: euclidean_pearson
|
|
value: 85.62978165451675
|
|
- type: euclidean_spearman
|
|
value: 86.54346234593214
|
|
- type: main_score
|
|
value: 86.54345663501209
|
|
- type: manhattan_pearson
|
|
value: 85.3032964455555
|
|
- type: manhattan_spearman
|
|
value: 86.30088652823572
|
|
- type: pearson
|
|
value: 84.13791770600066
|
|
- type: spearman
|
|
value: 86.54345663501209
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB STS16
|
|
revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
|
|
split: test
|
|
type: mteb/sts16-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 84.40315982722548
|
|
- type: cosine_spearman
|
|
value: 85.40751435377788
|
|
- type: euclidean_pearson
|
|
value: 84.35271010578505
|
|
- type: euclidean_spearman
|
|
value: 85.40751373941698
|
|
- type: main_score
|
|
value: 85.40751435377788
|
|
- type: manhattan_pearson
|
|
value: 84.17785174793401
|
|
- type: manhattan_spearman
|
|
value: 85.23156904732424
|
|
- type: pearson
|
|
value: 84.40315982722548
|
|
- type: spearman
|
|
value: 85.40751435377788
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: en-ar
|
|
name: MTEB STS17 (en-ar)
|
|
revision: faeb762787bd10488a50c8b5be4a3b82e411949c
|
|
split: test
|
|
type: mteb/sts17-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 59.98924365555529
|
|
- type: cosine_spearman
|
|
value: 60.12821686053337
|
|
- type: euclidean_pearson
|
|
value: 60.90431312863765
|
|
- type: euclidean_spearman
|
|
value: 60.12821686053337
|
|
- type: main_score
|
|
value: 60.12821686053337
|
|
- type: manhattan_pearson
|
|
value: 59.05369093717122
|
|
- type: manhattan_spearman
|
|
value: 57.65837693471568
|
|
- type: pearson
|
|
value: 59.98924365555529
|
|
- type: spearman
|
|
value: 60.12821686053337
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: en-de
|
|
name: MTEB STS17 (en-de)
|
|
revision: faeb762787bd10488a50c8b5be4a3b82e411949c
|
|
split: test
|
|
type: mteb/sts17-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 74.95271349225828
|
|
- type: cosine_spearman
|
|
value: 75.43839974308261
|
|
- type: euclidean_pearson
|
|
value: 75.68179466828151
|
|
- type: euclidean_spearman
|
|
value: 75.43839974308261
|
|
- type: main_score
|
|
value: 75.43839974308261
|
|
- type: manhattan_pearson
|
|
value: 75.4848070012919
|
|
- type: manhattan_spearman
|
|
value: 74.92507658877852
|
|
- type: pearson
|
|
value: 74.95271349225828
|
|
- type: spearman
|
|
value: 75.43839974308261
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: en-en
|
|
name: MTEB STS17 (en-en)
|
|
revision: faeb762787bd10488a50c8b5be4a3b82e411949c
|
|
split: test
|
|
type: mteb/sts17-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 86.18555151297676
|
|
- type: cosine_spearman
|
|
value: 86.40304228488033
|
|
- type: euclidean_pearson
|
|
value: 86.8788548303146
|
|
- type: euclidean_spearman
|
|
value: 86.40304228488033
|
|
- type: main_score
|
|
value: 86.40304228488033
|
|
- type: manhattan_pearson
|
|
value: 86.79312171236047
|
|
- type: manhattan_spearman
|
|
value: 86.26008520753594
|
|
- type: pearson
|
|
value: 86.18555151297676
|
|
- type: spearman
|
|
value: 86.40304228488033
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: en-tr
|
|
name: MTEB STS17 (en-tr)
|
|
revision: faeb762787bd10488a50c8b5be4a3b82e411949c
|
|
split: test
|
|
type: mteb/sts17-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 54.99479996647493
|
|
- type: cosine_spearman
|
|
value: 53.67766339389046
|
|
- type: euclidean_pearson
|
|
value: 55.32473081178422
|
|
- type: euclidean_spearman
|
|
value: 53.67766339389046
|
|
- type: main_score
|
|
value: 53.67766339389046
|
|
- type: manhattan_pearson
|
|
value: 54.66604584985125
|
|
- type: manhattan_spearman
|
|
value: 52.48322788533404
|
|
- type: pearson
|
|
value: 54.99479996647493
|
|
- type: spearman
|
|
value: 53.67766339389046
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: es-en
|
|
name: MTEB STS17 (es-en)
|
|
revision: faeb762787bd10488a50c8b5be4a3b82e411949c
|
|
split: test
|
|
type: mteb/sts17-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 76.65184590191937
|
|
- type: cosine_spearman
|
|
value: 78.04569100389011
|
|
- type: euclidean_pearson
|
|
value: 77.11425698246029
|
|
- type: euclidean_spearman
|
|
value: 78.04569100389011
|
|
- type: main_score
|
|
value: 78.04569100389011
|
|
- type: manhattan_pearson
|
|
value: 77.34799982307821
|
|
- type: manhattan_spearman
|
|
value: 78.22975685912238
|
|
- type: pearson
|
|
value: 76.65184590191937
|
|
- type: spearman
|
|
value: 78.04569100389011
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: fr-en
|
|
name: MTEB STS17 (fr-en)
|
|
revision: faeb762787bd10488a50c8b5be4a3b82e411949c
|
|
split: test
|
|
type: mteb/sts17-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 76.30743924244035
|
|
- type: cosine_spearman
|
|
value: 75.2110676227775
|
|
- type: euclidean_pearson
|
|
value: 77.10837892816058
|
|
- type: euclidean_spearman
|
|
value: 75.2110676227775
|
|
- type: main_score
|
|
value: 75.2110676227775
|
|
- type: manhattan_pearson
|
|
value: 76.814009334774
|
|
- type: manhattan_spearman
|
|
value: 74.96159426113054
|
|
- type: pearson
|
|
value: 76.30743924244035
|
|
- type: spearman
|
|
value: 75.2110676227775
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: it-en
|
|
name: MTEB STS17 (it-en)
|
|
revision: faeb762787bd10488a50c8b5be4a3b82e411949c
|
|
split: test
|
|
type: mteb/sts17-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 75.11771819741416
|
|
- type: cosine_spearman
|
|
value: 74.96778304560281
|
|
- type: euclidean_pearson
|
|
value: 75.56941540554674
|
|
- type: euclidean_spearman
|
|
value: 74.96778304560281
|
|
- type: main_score
|
|
value: 74.96778304560281
|
|
- type: manhattan_pearson
|
|
value: 75.18422319871718
|
|
- type: manhattan_spearman
|
|
value: 74.45788102060328
|
|
- type: pearson
|
|
value: 75.11771819741416
|
|
- type: spearman
|
|
value: 74.96778304560281
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: nl-en
|
|
name: MTEB STS17 (nl-en)
|
|
revision: faeb762787bd10488a50c8b5be4a3b82e411949c
|
|
split: test
|
|
type: mteb/sts17-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 72.42454093118816
|
|
- type: cosine_spearman
|
|
value: 71.9097547231894
|
|
- type: euclidean_pearson
|
|
value: 73.04051728705643
|
|
- type: euclidean_spearman
|
|
value: 71.9097547231894
|
|
- type: main_score
|
|
value: 71.9097547231894
|
|
- type: manhattan_pearson
|
|
value: 72.5487755597775
|
|
- type: manhattan_spearman
|
|
value: 71.080265405627
|
|
- type: pearson
|
|
value: 72.42454093118816
|
|
- type: spearman
|
|
value: 71.9097547231894
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: en
|
|
name: MTEB STS22 (en)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 69.3881685924264
|
|
- type: cosine_spearman
|
|
value: 69.37162939123382
|
|
- type: euclidean_pearson
|
|
value: 70.5377770359738
|
|
- type: euclidean_spearman
|
|
value: 69.37162939123382
|
|
- type: main_score
|
|
value: 69.37162939123382
|
|
- type: manhattan_pearson
|
|
value: 70.86501303890763
|
|
- type: manhattan_spearman
|
|
value: 69.54018077011284
|
|
- type: pearson
|
|
value: 69.3881685924264
|
|
- type: spearman
|
|
value: 69.37162939123382
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: de-en
|
|
name: MTEB STS22 (de-en)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 64.64985744446284
|
|
- type: cosine_spearman
|
|
value: 63.89323074678119
|
|
- type: euclidean_pearson
|
|
value: 66.9623010036117
|
|
- type: euclidean_spearman
|
|
value: 63.89323074678119
|
|
- type: main_score
|
|
value: 63.89323074678119
|
|
- type: manhattan_pearson
|
|
value: 68.60076281156398
|
|
- type: manhattan_spearman
|
|
value: 64.80183430943912
|
|
- type: pearson
|
|
value: 64.64985744446284
|
|
- type: spearman
|
|
value: 63.89323074678119
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: es-en
|
|
name: MTEB STS22 (es-en)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 78.95094282575697
|
|
- type: cosine_spearman
|
|
value: 80.66341954222823
|
|
- type: euclidean_pearson
|
|
value: 79.7677956183949
|
|
- type: euclidean_spearman
|
|
value: 80.66341954222823
|
|
- type: main_score
|
|
value: 80.66341954222823
|
|
- type: manhattan_pearson
|
|
value: 81.52201735972797
|
|
- type: manhattan_spearman
|
|
value: 81.65309541429473
|
|
- type: pearson
|
|
value: 78.95094282575697
|
|
- type: spearman
|
|
value: 80.66341954222823
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: pl-en
|
|
name: MTEB STS22 (pl-en)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 77.99167158750629
|
|
- type: cosine_spearman
|
|
value: 77.00326330683939
|
|
- type: euclidean_pearson
|
|
value: 77.60571751826936
|
|
- type: euclidean_spearman
|
|
value: 77.00326330683939
|
|
- type: main_score
|
|
value: 77.00326330683939
|
|
- type: manhattan_pearson
|
|
value: 78.19839585217989
|
|
- type: manhattan_spearman
|
|
value: 78.44894390841364
|
|
- type: pearson
|
|
value: 77.99167158750629
|
|
- type: spearman
|
|
value: 77.00326330683939
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: zh-en
|
|
name: MTEB STS22 (zh-en)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 77.21035942564082
|
|
- type: cosine_spearman
|
|
value: 76.57212143103963
|
|
- type: euclidean_pearson
|
|
value: 78.03973868360728
|
|
- type: euclidean_spearman
|
|
value: 76.57212143103963
|
|
- type: main_score
|
|
value: 76.57212143103963
|
|
- type: manhattan_pearson
|
|
value: 78.16591898142042
|
|
- type: manhattan_spearman
|
|
value: 76.83958214147293
|
|
- type: pearson
|
|
value: 77.21035942564082
|
|
- type: spearman
|
|
value: 76.57212143103963
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB STSBenchmark
|
|
revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
|
|
split: test
|
|
type: mteb/stsbenchmark-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 81.21615375003084
|
|
- type: cosine_spearman
|
|
value: 84.2970803211202
|
|
- type: euclidean_pearson
|
|
value: 83.54765755364517
|
|
- type: euclidean_spearman
|
|
value: 84.2970803211202
|
|
- type: main_score
|
|
value: 84.2970803211202
|
|
- type: manhattan_pearson
|
|
value: 83.2769664077453
|
|
- type: manhattan_spearman
|
|
value: 84.09545601307758
|
|
- type: pearson
|
|
value: 81.21615375003084
|
|
- type: spearman
|
|
value: 84.2970803211202
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SciDocsRR
|
|
revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
|
|
split: test
|
|
type: mteb/scidocs-reranking
|
|
metrics:
|
|
- type: main_score
|
|
value: 80.72245608609909
|
|
- type: map
|
|
value: 80.72245608609909
|
|
- type: mrr
|
|
value: 94.86804408373035
|
|
- type: nAUC_map_diff1
|
|
value: 3.565293868431913
|
|
- type: nAUC_map_max
|
|
value: 53.87118155384518
|
|
- type: nAUC_map_std
|
|
value: 69.73850807835032
|
|
- type: nAUC_mrr_diff1
|
|
value: 48.33938058863373
|
|
- type: nAUC_mrr_max
|
|
value: 82.0796869926262
|
|
- type: nAUC_mrr_std
|
|
value: 79.20228314778093
|
|
task:
|
|
type: Reranking
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SciFact
|
|
revision: 0228b52cf27578f30900b9e5271d331663a030d7
|
|
split: test
|
|
type: mteb/scifact
|
|
metrics:
|
|
- type: main_score
|
|
value: 72.604
|
|
- type: map_at_1
|
|
value: 57.05
|
|
- type: map_at_10
|
|
value: 68.026
|
|
- type: map_at_100
|
|
value: 68.54299999999999
|
|
- type: map_at_1000
|
|
value: 68.56
|
|
- type: map_at_20
|
|
value: 68.329
|
|
- type: map_at_3
|
|
value: 65.565
|
|
- type: map_at_5
|
|
value: 66.81899999999999
|
|
- type: mrr_at_1
|
|
value: 60.0
|
|
- type: mrr_at_10
|
|
value: 68.97116402116401
|
|
- type: mrr_at_100
|
|
value: 69.43171438050388
|
|
- type: mrr_at_1000
|
|
value: 69.44900642374887
|
|
- type: mrr_at_20
|
|
value: 69.25799802049801
|
|
- type: mrr_at_3
|
|
value: 67.11111111111111
|
|
- type: mrr_at_5
|
|
value: 68.27777777777779
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 66.45098144160822
|
|
- type: nauc_map_at_1000_max
|
|
value: 52.26713946112144
|
|
- type: nauc_map_at_1000_std
|
|
value: -3.2435941711161194
|
|
- type: nauc_map_at_100_diff1
|
|
value: 66.45069255591629
|
|
- type: nauc_map_at_100_max
|
|
value: 52.277529223166994
|
|
- type: nauc_map_at_100_std
|
|
value: -3.236289003540743
|
|
- type: nauc_map_at_10_diff1
|
|
value: 66.50847900934123
|
|
- type: nauc_map_at_10_max
|
|
value: 52.56336813799116
|
|
- type: nauc_map_at_10_std
|
|
value: -3.2225840417202547
|
|
- type: nauc_map_at_1_diff1
|
|
value: 69.8066007922827
|
|
- type: nauc_map_at_1_max
|
|
value: 46.19700236373352
|
|
- type: nauc_map_at_1_std
|
|
value: -11.167127232139137
|
|
- type: nauc_map_at_20_diff1
|
|
value: 66.49775686319742
|
|
- type: nauc_map_at_20_max
|
|
value: 52.31488178119375
|
|
- type: nauc_map_at_20_std
|
|
value: -3.528866881477926
|
|
- type: nauc_map_at_3_diff1
|
|
value: 67.0124735448113
|
|
- type: nauc_map_at_3_max
|
|
value: 51.47207513467635
|
|
- type: nauc_map_at_3_std
|
|
value: -4.688170694240992
|
|
- type: nauc_map_at_5_diff1
|
|
value: 66.37338579400031
|
|
- type: nauc_map_at_5_max
|
|
value: 51.03182506884805
|
|
- type: nauc_map_at_5_std
|
|
value: -4.090110073585303
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 66.13316468798861
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 53.18661162667272
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -1.1549432899803578
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 66.13308912088833
|
|
- type: nauc_mrr_at_100_max
|
|
value: 53.196523181344176
|
|
- type: nauc_mrr_at_100_std
|
|
value: -1.148961396684306
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 66.11198414850364
|
|
- type: nauc_mrr_at_10_max
|
|
value: 53.45434553493992
|
|
- type: nauc_mrr_at_10_std
|
|
value: -1.0202103385535555
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 69.18818640546156
|
|
- type: nauc_mrr_at_1_max
|
|
value: 50.224102107450285
|
|
- type: nauc_mrr_at_1_std
|
|
value: -4.4508756307510104
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 66.12038286624204
|
|
- type: nauc_mrr_at_20_max
|
|
value: 53.23900442821744
|
|
- type: nauc_mrr_at_20_std
|
|
value: -1.3453691424031584
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 66.23482655095762
|
|
- type: nauc_mrr_at_3_max
|
|
value: 53.519304370411625
|
|
- type: nauc_mrr_at_3_std
|
|
value: -1.0512555098049736
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 65.63605277411375
|
|
- type: nauc_mrr_at_5_max
|
|
value: 53.17390536531564
|
|
- type: nauc_mrr_at_5_std
|
|
value: -0.5198682324341892
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 65.85075826609345
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 53.814329968179045
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -0.9856729250792472
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 65.78229528993444
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 54.1747645815977
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -0.47502756295876847
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 66.00876580480991
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 55.06235713538037
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -1.5534145585575012
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 69.18818640546156
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 50.224102107450285
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -4.4508756307510104
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 65.95831573232856
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 54.24206688010573
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -2.705254164112238
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 66.14046065126678
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 54.07332075118414
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -2.0119140501882793
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 65.21102868019805
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 52.596880916483165
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -2.1720193236802023
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -21.99504940846271
|
|
- type: nauc_precision_at_1000_max
|
|
value: 19.25403291298791
|
|
- type: nauc_precision_at_1000_std
|
|
value: 46.296476764054404
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -11.741691903205695
|
|
- type: nauc_precision_at_100_max
|
|
value: 25.699636707900623
|
|
- type: nauc_precision_at_100_std
|
|
value: 43.96233624765463
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 11.568895847591932
|
|
- type: nauc_precision_at_10_max
|
|
value: 39.43006347212197
|
|
- type: nauc_precision_at_10_std
|
|
value: 28.751839941496836
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 69.18818640546156
|
|
- type: nauc_precision_at_1_max
|
|
value: 50.224102107450285
|
|
- type: nauc_precision_at_1_std
|
|
value: -4.4508756307510104
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 4.854833212085455
|
|
- type: nauc_precision_at_20_max
|
|
value: 34.19851755381116
|
|
- type: nauc_precision_at_20_std
|
|
value: 28.728626880402068
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 35.04823458092479
|
|
- type: nauc_precision_at_3_max
|
|
value: 47.8670338954734
|
|
- type: nauc_precision_at_3_std
|
|
value: 19.389299130775157
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 25.605002849466736
|
|
- type: nauc_precision_at_5_max
|
|
value: 43.50575999348689
|
|
- type: nauc_precision_at_5_std
|
|
value: 24.80257266140189
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 55.07703081232429
|
|
- type: nauc_recall_at_1000_max
|
|
value: 70.71661998132596
|
|
- type: nauc_recall_at_1000_std
|
|
value: 64.58916900093288
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 59.97732426303837
|
|
- type: nauc_recall_at_100_max
|
|
value: 71.64532479658504
|
|
- type: nauc_recall_at_100_std
|
|
value: 37.87515006002412
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 64.45621875630812
|
|
- type: nauc_recall_at_10_max
|
|
value: 64.72171592433827
|
|
- type: nauc_recall_at_10_std
|
|
value: 0.9026532647803642
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 69.8066007922827
|
|
- type: nauc_recall_at_1_max
|
|
value: 46.19700236373352
|
|
- type: nauc_recall_at_1_std
|
|
value: -11.167127232139137
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 63.79448821637328
|
|
- type: nauc_recall_at_20_max
|
|
value: 61.597381158568524
|
|
- type: nauc_recall_at_20_std
|
|
value: -7.27449509788767
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 64.75442031192492
|
|
- type: nauc_recall_at_3_max
|
|
value: 56.12106077054382
|
|
- type: nauc_recall_at_3_std
|
|
value: -2.661587128227682
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 60.82940800383688
|
|
- type: nauc_recall_at_5_max
|
|
value: 53.647222430433736
|
|
- type: nauc_recall_at_5_std
|
|
value: -0.793229884870239
|
|
- type: ndcg_at_1
|
|
value: 60.0
|
|
- type: ndcg_at_10
|
|
value: 72.604
|
|
- type: ndcg_at_100
|
|
value: 74.83800000000001
|
|
- type: ndcg_at_1000
|
|
value: 75.27199999999999
|
|
- type: ndcg_at_20
|
|
value: 73.599
|
|
- type: ndcg_at_3
|
|
value: 68.509
|
|
- type: ndcg_at_5
|
|
value: 70.352
|
|
- type: precision_at_1
|
|
value: 60.0
|
|
- type: precision_at_10
|
|
value: 9.733
|
|
- type: precision_at_100
|
|
value: 1.083
|
|
- type: precision_at_1000
|
|
value: 0.11199999999999999
|
|
- type: precision_at_20
|
|
value: 5.067
|
|
- type: precision_at_3
|
|
value: 27.444000000000003
|
|
- type: precision_at_5
|
|
value: 17.666999999999998
|
|
- type: recall_at_1
|
|
value: 57.05
|
|
- type: recall_at_10
|
|
value: 85.422
|
|
- type: recall_at_100
|
|
value: 95.333
|
|
- type: recall_at_1000
|
|
value: 98.667
|
|
- type: recall_at_20
|
|
value: 89.156
|
|
- type: recall_at_3
|
|
value: 74.211
|
|
- type: recall_at_5
|
|
value: 79.094
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SprintDuplicateQuestions
|
|
revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
|
|
split: test
|
|
type: mteb/sprintduplicatequestions-pairclassification
|
|
metrics:
|
|
- type: cosine_accuracy
|
|
value: 99.76237623762376
|
|
- type: cosine_accuracy_threshold
|
|
value: 89.08973932266235
|
|
- type: cosine_ap
|
|
value: 93.82184396471453
|
|
- type: cosine_f1
|
|
value: 87.87878787878789
|
|
- type: cosine_f1_threshold
|
|
value: 89.08973932266235
|
|
- type: cosine_precision
|
|
value: 88.77551020408163
|
|
- type: cosine_recall
|
|
value: 87.0
|
|
- type: dot_accuracy
|
|
value: 99.76237623762376
|
|
- type: dot_accuracy_threshold
|
|
value: 89.08973932266235
|
|
- type: dot_ap
|
|
value: 93.82179339271785
|
|
- type: dot_f1
|
|
value: 87.87878787878789
|
|
- type: dot_f1_threshold
|
|
value: 89.08973932266235
|
|
- type: dot_precision
|
|
value: 88.77551020408163
|
|
- type: dot_recall
|
|
value: 87.0
|
|
- type: euclidean_accuracy
|
|
value: 99.76237623762376
|
|
- type: euclidean_accuracy_threshold
|
|
value: 46.71244025230408
|
|
- type: euclidean_ap
|
|
value: 93.82184396471453
|
|
- type: euclidean_f1
|
|
value: 87.87878787878789
|
|
- type: euclidean_f1_threshold
|
|
value: 46.71244025230408
|
|
- type: euclidean_precision
|
|
value: 88.77551020408163
|
|
- type: euclidean_recall
|
|
value: 87.0
|
|
- type: main_score
|
|
value: 94.18170827750167
|
|
- type: manhattan_accuracy
|
|
value: 99.77425742574258
|
|
- type: manhattan_accuracy_threshold
|
|
value: 1095.131492614746
|
|
- type: manhattan_ap
|
|
value: 94.18170827750167
|
|
- type: manhattan_f1
|
|
value: 88.45577211394303
|
|
- type: manhattan_f1_threshold
|
|
value: 1108.85648727417
|
|
- type: manhattan_precision
|
|
value: 88.41158841158841
|
|
- type: manhattan_recall
|
|
value: 88.5
|
|
- type: max_ap
|
|
value: 94.18170827750167
|
|
- type: max_f1
|
|
value: 88.45577211394303
|
|
- type: max_precision
|
|
value: 88.77551020408163
|
|
- type: max_recall
|
|
value: 88.5
|
|
- type: similarity_accuracy
|
|
value: 99.76237623762376
|
|
- type: similarity_accuracy_threshold
|
|
value: 89.08973932266235
|
|
- type: similarity_ap
|
|
value: 93.82184396471453
|
|
- type: similarity_f1
|
|
value: 87.87878787878789
|
|
- type: similarity_f1_threshold
|
|
value: 89.08973932266235
|
|
- type: similarity_precision
|
|
value: 88.77551020408163
|
|
- type: similarity_recall
|
|
value: 87.0
|
|
task:
|
|
type: PairClassification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB StackExchangeClustering
|
|
revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
|
|
split: test
|
|
type: mteb/stackexchange-clustering
|
|
metrics:
|
|
- type: main_score
|
|
value: 65.93583959980214
|
|
- type: v_measure
|
|
value: 65.93583959980214
|
|
- type: v_measure_std
|
|
value: 3.9403815544270233
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB StackExchangeClusteringP2P
|
|
revision: 815ca46b2622cec33ccafc3735d572c266efdb44
|
|
split: test
|
|
type: mteb/stackexchange-clustering-p2p
|
|
metrics:
|
|
- type: main_score
|
|
value: 35.594885571404724
|
|
- type: v_measure
|
|
value: 35.594885571404724
|
|
- type: v_measure_std
|
|
value: 1.5163847345337254
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB StackOverflowDupQuestions
|
|
revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
|
|
split: test
|
|
type: mteb/stackoverflowdupquestions-reranking
|
|
metrics:
|
|
- type: main_score
|
|
value: 49.85213562933509
|
|
- type: map
|
|
value: 49.85213562933509
|
|
- type: mrr
|
|
value: 50.62702922077922
|
|
- type: nAUC_map_diff1
|
|
value: 36.55011836042864
|
|
- type: nAUC_map_max
|
|
value: 13.45991062036654
|
|
- type: nAUC_map_std
|
|
value: 10.192881915639742
|
|
- type: nAUC_mrr_diff1
|
|
value: 37.058265888016976
|
|
- type: nAUC_mrr_max
|
|
value: 14.081819232783383
|
|
- type: nAUC_mrr_std
|
|
value: 11.215978874656958
|
|
task:
|
|
type: Reranking
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SummEval
|
|
revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
|
|
split: test
|
|
type: mteb/summeval
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 25.349220308622627
|
|
- type: cosine_spearman
|
|
value: 27.880975911253458
|
|
- type: dot_pearson
|
|
value: 25.349197273883224
|
|
- type: dot_spearman
|
|
value: 27.880903951553655
|
|
- type: main_score
|
|
value: 27.880975911253458
|
|
- type: pearson
|
|
value: 25.349220308622627
|
|
- type: spearman
|
|
value: 27.880975911253458
|
|
task:
|
|
type: Summarization
|
|
- dataset:
|
|
config: default
|
|
name: MTEB TRECCOVID
|
|
revision: bb9466bac8153a0349341eb1b22e06409e78ef4e
|
|
split: test
|
|
type: mteb/trec-covid
|
|
metrics:
|
|
- type: main_score
|
|
value: 79.536
|
|
- type: map_at_1
|
|
value: 0.215
|
|
- type: map_at_10
|
|
value: 2.048
|
|
- type: map_at_100
|
|
value: 12.842999999999998
|
|
- type: map_at_1000
|
|
value: 31.032
|
|
- type: map_at_20
|
|
value: 3.8379999999999996
|
|
- type: map_at_3
|
|
value: 0.64
|
|
- type: map_at_5
|
|
value: 1.052
|
|
- type: mrr_at_1
|
|
value: 84.0
|
|
- type: mrr_at_10
|
|
value: 91.16666666666666
|
|
- type: mrr_at_100
|
|
value: 91.16666666666666
|
|
- type: mrr_at_1000
|
|
value: 91.16666666666666
|
|
- type: mrr_at_20
|
|
value: 91.16666666666666
|
|
- type: mrr_at_3
|
|
value: 90.66666666666666
|
|
- type: mrr_at_5
|
|
value: 91.16666666666666
|
|
- type: nauc_map_at_1000_diff1
|
|
value: -18.530580290412697
|
|
- type: nauc_map_at_1000_max
|
|
value: 43.14744028154331
|
|
- type: nauc_map_at_1000_std
|
|
value: 79.6699665194256
|
|
- type: nauc_map_at_100_diff1
|
|
value: -21.271315814062437
|
|
- type: nauc_map_at_100_max
|
|
value: 17.55081814849073
|
|
- type: nauc_map_at_100_std
|
|
value: 48.17729810787553
|
|
- type: nauc_map_at_10_diff1
|
|
value: -11.002124943974252
|
|
- type: nauc_map_at_10_max
|
|
value: -9.6495971981689
|
|
- type: nauc_map_at_10_std
|
|
value: 6.648364965330221
|
|
- type: nauc_map_at_1_diff1
|
|
value: 0.1251393811417004
|
|
- type: nauc_map_at_1_max
|
|
value: -12.601700488498643
|
|
- type: nauc_map_at_1_std
|
|
value: -3.5018878780762366
|
|
- type: nauc_map_at_20_diff1
|
|
value: -19.526191160714987
|
|
- type: nauc_map_at_20_max
|
|
value: -4.175483070077258
|
|
- type: nauc_map_at_20_std
|
|
value: 16.014345473073693
|
|
- type: nauc_map_at_3_diff1
|
|
value: -0.8632406748675692
|
|
- type: nauc_map_at_3_max
|
|
value: -12.9654502212951
|
|
- type: nauc_map_at_3_std
|
|
value: -1.5551804410996426
|
|
- type: nauc_map_at_5_diff1
|
|
value: -9.294941718115151
|
|
- type: nauc_map_at_5_max
|
|
value: -12.795655812948572
|
|
- type: nauc_map_at_5_std
|
|
value: 0.6128051906803516
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 33.997935217447434
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 41.160149696734955
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 27.657024869568446
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 33.997935217447434
|
|
- type: nauc_mrr_at_100_max
|
|
value: 41.160149696734955
|
|
- type: nauc_mrr_at_100_std
|
|
value: 27.657024869568446
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 33.997935217447434
|
|
- type: nauc_mrr_at_10_max
|
|
value: 41.160149696734955
|
|
- type: nauc_mrr_at_10_std
|
|
value: 27.657024869568446
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 37.279086892488884
|
|
- type: nauc_mrr_at_1_max
|
|
value: 43.292832596956316
|
|
- type: nauc_mrr_at_1_std
|
|
value: 20.305596465390227
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 33.997935217447434
|
|
- type: nauc_mrr_at_20_max
|
|
value: 41.160149696734955
|
|
- type: nauc_mrr_at_20_std
|
|
value: 27.657024869568446
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 31.138610414926326
|
|
- type: nauc_mrr_at_3_max
|
|
value: 39.545043163464186
|
|
- type: nauc_mrr_at_3_std
|
|
value: 31.70252018936244
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 33.997935217447434
|
|
- type: nauc_mrr_at_5_max
|
|
value: 41.160149696734955
|
|
- type: nauc_mrr_at_5_std
|
|
value: 27.657024869568446
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: -20.948326611476556
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 36.766927406101956
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 75.32635798841658
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: -14.54815381092273
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 51.38801585344711
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 76.47002281413397
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: -12.80351464937073
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 35.71831279387225
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 52.15347275643156
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 20.42160737812909
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 34.20619235836624
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 13.088179936005965
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: -18.116251292365128
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 46.9808896232964
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 61.73761431506857
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: -4.44558396286013
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 26.953553278525938
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 33.375410187254786
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: -15.495190925371652
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 29.21035888164427
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 41.168078957076396
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 6.339888107354097
|
|
- type: nauc_precision_at_1000_max
|
|
value: 51.87294743895088
|
|
- type: nauc_precision_at_1000_std
|
|
value: 49.22667294372217
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -10.245901160105356
|
|
- type: nauc_precision_at_100_max
|
|
value: 56.07707608097002
|
|
- type: nauc_precision_at_100_std
|
|
value: 78.96626562096216
|
|
- type: nauc_precision_at_10_diff1
|
|
value: -4.590219332829025
|
|
- type: nauc_precision_at_10_max
|
|
value: 47.52908614003191
|
|
- type: nauc_precision_at_10_std
|
|
value: 59.53043786106239
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 37.279086892488884
|
|
- type: nauc_precision_at_1_max
|
|
value: 43.292832596956316
|
|
- type: nauc_precision_at_1_std
|
|
value: 20.305596465390227
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -14.763079024242392
|
|
- type: nauc_precision_at_20_max
|
|
value: 56.25820402898436
|
|
- type: nauc_precision_at_20_std
|
|
value: 67.6952843431086
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 2.9292734630949067
|
|
- type: nauc_precision_at_3_max
|
|
value: 41.296148445888285
|
|
- type: nauc_precision_at_3_std
|
|
value: 46.551771604768255
|
|
- type: nauc_precision_at_5_diff1
|
|
value: -15.368719472623535
|
|
- type: nauc_precision_at_5_max
|
|
value: 39.706937186186984
|
|
- type: nauc_precision_at_5_std
|
|
value: 45.991734125764275
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: -18.70157967410686
|
|
- type: nauc_recall_at_1000_max
|
|
value: 27.303031147629746
|
|
- type: nauc_recall_at_1000_std
|
|
value: 63.59247900235757
|
|
- type: nauc_recall_at_100_diff1
|
|
value: -21.505202598262795
|
|
- type: nauc_recall_at_100_max
|
|
value: 3.1053955846040666
|
|
- type: nauc_recall_at_100_std
|
|
value: 35.59388419574821
|
|
- type: nauc_recall_at_10_diff1
|
|
value: -13.309140466736356
|
|
- type: nauc_recall_at_10_max
|
|
value: -16.90482412154473
|
|
- type: nauc_recall_at_10_std
|
|
value: 2.1355678490728542
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 0.1251393811417004
|
|
- type: nauc_recall_at_1_max
|
|
value: -12.601700488498643
|
|
- type: nauc_recall_at_1_std
|
|
value: -3.5018878780762366
|
|
- type: nauc_recall_at_20_diff1
|
|
value: -21.303497421292096
|
|
- type: nauc_recall_at_20_max
|
|
value: -13.765429909809388
|
|
- type: nauc_recall_at_20_std
|
|
value: 9.07482009539061
|
|
- type: nauc_recall_at_3_diff1
|
|
value: -6.017177782774693
|
|
- type: nauc_recall_at_3_max
|
|
value: -19.064966459546255
|
|
- type: nauc_recall_at_3_std
|
|
value: -3.0227410013796967
|
|
- type: nauc_recall_at_5_diff1
|
|
value: -14.078289790672653
|
|
- type: nauc_recall_at_5_max
|
|
value: -19.52038684292809
|
|
- type: nauc_recall_at_5_std
|
|
value: -2.6267198328675994
|
|
- type: ndcg_at_1
|
|
value: 78.0
|
|
- type: ndcg_at_10
|
|
value: 79.536
|
|
- type: ndcg_at_100
|
|
value: 62.65500000000001
|
|
- type: ndcg_at_1000
|
|
value: 56.359
|
|
- type: ndcg_at_20
|
|
value: 77.561
|
|
- type: ndcg_at_3
|
|
value: 80.296
|
|
- type: ndcg_at_5
|
|
value: 79.806
|
|
- type: precision_at_1
|
|
value: 84.0
|
|
- type: precision_at_10
|
|
value: 85.6
|
|
- type: precision_at_100
|
|
value: 64.92
|
|
- type: precision_at_1000
|
|
value: 24.89
|
|
- type: precision_at_20
|
|
value: 83.2
|
|
- type: precision_at_3
|
|
value: 87.333
|
|
- type: precision_at_5
|
|
value: 87.2
|
|
- type: recall_at_1
|
|
value: 0.215
|
|
- type: recall_at_10
|
|
value: 2.246
|
|
- type: recall_at_100
|
|
value: 15.784
|
|
- type: recall_at_1000
|
|
value: 53.427
|
|
- type: recall_at_20
|
|
value: 4.281
|
|
- type: recall_at_3
|
|
value: 0.688
|
|
- type: recall_at_5
|
|
value: 1.142
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB Touche2020
|
|
revision: a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f
|
|
split: test
|
|
type: mteb/touche2020
|
|
metrics:
|
|
- type: main_score
|
|
value: 31.186999999999998
|
|
- type: map_at_1
|
|
value: 3.4070000000000005
|
|
- type: map_at_10
|
|
value: 13.313
|
|
- type: map_at_100
|
|
value: 19.900000000000002
|
|
- type: map_at_1000
|
|
value: 21.437
|
|
- type: map_at_20
|
|
value: 15.714
|
|
- type: map_at_3
|
|
value: 6.923
|
|
- type: map_at_5
|
|
value: 9.054
|
|
- type: mrr_at_1
|
|
value: 44.89795918367347
|
|
- type: mrr_at_10
|
|
value: 56.63832199546485
|
|
- type: mrr_at_100
|
|
value: 57.666166033512965
|
|
- type: mrr_at_1000
|
|
value: 57.666166033512965
|
|
- type: mrr_at_20
|
|
value: 57.51229496127455
|
|
- type: mrr_at_3
|
|
value: 53.40136054421768
|
|
- type: mrr_at_5
|
|
value: 55.1360544217687
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 6.929929189103678
|
|
- type: nauc_map_at_1000_max
|
|
value: -20.5925373398606
|
|
- type: nauc_map_at_1000_std
|
|
value: 7.835669658058121
|
|
- type: nauc_map_at_100_diff1
|
|
value: 7.528899533894891
|
|
- type: nauc_map_at_100_max
|
|
value: -21.032199268806018
|
|
- type: nauc_map_at_100_std
|
|
value: 5.370650925959299
|
|
- type: nauc_map_at_10_diff1
|
|
value: 14.176770339374578
|
|
- type: nauc_map_at_10_max
|
|
value: -19.194036092916633
|
|
- type: nauc_map_at_10_std
|
|
value: -14.964801890692026
|
|
- type: nauc_map_at_1_diff1
|
|
value: 16.059944358241733
|
|
- type: nauc_map_at_1_max
|
|
value: -25.302527766801695
|
|
- type: nauc_map_at_1_std
|
|
value: -13.565207797491604
|
|
- type: nauc_map_at_20_diff1
|
|
value: 11.361043123465297
|
|
- type: nauc_map_at_20_max
|
|
value: -18.0301938420575
|
|
- type: nauc_map_at_20_std
|
|
value: -7.25573010108597
|
|
- type: nauc_map_at_3_diff1
|
|
value: 21.973707928327727
|
|
- type: nauc_map_at_3_max
|
|
value: -20.079194093834058
|
|
- type: nauc_map_at_3_std
|
|
value: -20.173080790091422
|
|
- type: nauc_map_at_5_diff1
|
|
value: 19.669071376698206
|
|
- type: nauc_map_at_5_max
|
|
value: -23.679751632414845
|
|
- type: nauc_map_at_5_std
|
|
value: -20.28001860761147
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 6.875737447320781
|
|
- type: nauc_mrr_at_1000_max
|
|
value: -44.8769334243922
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 7.361962913444513
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 6.875737447320781
|
|
- type: nauc_mrr_at_100_max
|
|
value: -44.8769334243922
|
|
- type: nauc_mrr_at_100_std
|
|
value: 7.361962913444513
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 6.574806453972689
|
|
- type: nauc_mrr_at_10_max
|
|
value: -47.267277277496596
|
|
- type: nauc_mrr_at_10_std
|
|
value: 8.783148855636174
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 12.940754496022242
|
|
- type: nauc_mrr_at_1_max
|
|
value: -35.544013626458145
|
|
- type: nauc_mrr_at_1_std
|
|
value: 6.0616339439628915
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 7.179017109424859
|
|
- type: nauc_mrr_at_20_max
|
|
value: -45.52183055340191
|
|
- type: nauc_mrr_at_20_std
|
|
value: 6.960503593984209
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 2.10431985300728
|
|
- type: nauc_mrr_at_3_max
|
|
value: -41.662819302741184
|
|
- type: nauc_mrr_at_3_std
|
|
value: 5.68448693989341
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 5.25929369032379
|
|
- type: nauc_mrr_at_5_max
|
|
value: -44.62592534259141
|
|
- type: nauc_mrr_at_5_std
|
|
value: 6.26151671868977
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: -6.563466320842519
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: -33.15200693567147
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 29.09290649197198
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: -4.290185637900728
|
|
- type: nauc_ndcg_at_100_max
|
|
value: -35.6991058391752
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 24.47606141799262
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 4.171305930645993
|
|
- type: nauc_ndcg_at_10_max
|
|
value: -33.02156808389195
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -0.7115167969929295
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 4.295135743080979
|
|
- type: nauc_ndcg_at_1_max
|
|
value: -30.841816609035575
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 11.08702259742227
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 5.716130418772172
|
|
- type: nauc_ndcg_at_20_max
|
|
value: -32.02017772879846
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 0.42043490374547515
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 0.7696408676847266
|
|
- type: nauc_ndcg_at_3_max
|
|
value: -28.19446012238678
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 1.4270173161697919
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 4.011877087450832
|
|
- type: nauc_ndcg_at_5_max
|
|
value: -35.474817068811866
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -1.0183501951460643
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -18.852617887278956
|
|
- type: nauc_precision_at_1000_max
|
|
value: 26.536677685298997
|
|
- type: nauc_precision_at_1000_std
|
|
value: 31.17777014427175
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -21.993356262198738
|
|
- type: nauc_precision_at_100_max
|
|
value: -14.151354806872973
|
|
- type: nauc_precision_at_100_std
|
|
value: 68.01931004336306
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 3.518175306600991
|
|
- type: nauc_precision_at_10_max
|
|
value: -34.29876549408336
|
|
- type: nauc_precision_at_10_std
|
|
value: 8.571886047048881
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 12.940754496022242
|
|
- type: nauc_precision_at_1_max
|
|
value: -35.544013626458145
|
|
- type: nauc_precision_at_1_std
|
|
value: 6.0616339439628915
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 6.23454071647187
|
|
- type: nauc_precision_at_20_max
|
|
value: -29.16565290719762
|
|
- type: nauc_precision_at_20_std
|
|
value: 25.567483624610297
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 8.77511441582519
|
|
- type: nauc_precision_at_3_max
|
|
value: -29.389312907952135
|
|
- type: nauc_precision_at_3_std
|
|
value: -6.397150206890867
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 9.795445750266063
|
|
- type: nauc_precision_at_5_max
|
|
value: -38.88827845334236
|
|
- type: nauc_precision_at_5_std
|
|
value: -3.397760151003072
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: -28.033327034031043
|
|
- type: nauc_recall_at_1000_max
|
|
value: -15.30930042500693
|
|
- type: nauc_recall_at_1000_std
|
|
value: 69.27496829698434
|
|
- type: nauc_recall_at_100_diff1
|
|
value: -12.558500592244782
|
|
- type: nauc_recall_at_100_max
|
|
value: -27.109814142314832
|
|
- type: nauc_recall_at_100_std
|
|
value: 40.23660136119213
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 8.859020421080002
|
|
- type: nauc_recall_at_10_max
|
|
value: -26.101835112681034
|
|
- type: nauc_recall_at_10_std
|
|
value: -12.02508230851673
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 16.059944358241733
|
|
- type: nauc_recall_at_1_max
|
|
value: -25.302527766801695
|
|
- type: nauc_recall_at_1_std
|
|
value: -13.565207797491604
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 6.598503996413421
|
|
- type: nauc_recall_at_20_max
|
|
value: -25.661355219947264
|
|
- type: nauc_recall_at_20_std
|
|
value: -0.5270972932429998
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 15.848752699477423
|
|
- type: nauc_recall_at_3_max
|
|
value: -20.67227958185249
|
|
- type: nauc_recall_at_3_std
|
|
value: -19.687883601951533
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 15.210234895525055
|
|
- type: nauc_recall_at_5_max
|
|
value: -30.20253332454299
|
|
- type: nauc_recall_at_5_std
|
|
value: -19.986130369906242
|
|
- type: ndcg_at_1
|
|
value: 40.816
|
|
- type: ndcg_at_10
|
|
value: 31.186999999999998
|
|
- type: ndcg_at_100
|
|
value: 42.742000000000004
|
|
- type: ndcg_at_1000
|
|
value: 53.230999999999995
|
|
- type: ndcg_at_20
|
|
value: 31.057000000000002
|
|
- type: ndcg_at_3
|
|
value: 34.382000000000005
|
|
- type: ndcg_at_5
|
|
value: 32.038
|
|
- type: precision_at_1
|
|
value: 44.897999999999996
|
|
- type: precision_at_10
|
|
value: 27.143
|
|
- type: precision_at_100
|
|
value: 8.735
|
|
- type: precision_at_1000
|
|
value: 1.59
|
|
- type: precision_at_20
|
|
value: 19.898
|
|
- type: precision_at_3
|
|
value: 34.694
|
|
- type: precision_at_5
|
|
value: 31.019999999999996
|
|
- type: recall_at_1
|
|
value: 3.4070000000000005
|
|
- type: recall_at_10
|
|
value: 19.987
|
|
- type: recall_at_100
|
|
value: 52.888999999999996
|
|
- type: recall_at_1000
|
|
value: 85.172
|
|
- type: recall_at_20
|
|
value: 27.025
|
|
- type: recall_at_3
|
|
value: 7.774
|
|
- type: recall_at_5
|
|
value: 11.571
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB ToxicConversationsClassification
|
|
revision: edfaf9da55d3dd50d43143d90c1ac476895ae6de
|
|
split: test
|
|
type: mteb/toxic_conversations_50k
|
|
metrics:
|
|
- type: accuracy
|
|
value: 89.0380859375
|
|
- type: ap
|
|
value: 34.26536468203791
|
|
- type: ap_weighted
|
|
value: 34.26536468203791
|
|
- type: f1
|
|
value: 73.86921962038298
|
|
- type: f1_weighted
|
|
value: 90.61132302248866
|
|
- type: main_score
|
|
value: 89.0380859375
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB TweetSentimentExtractionClassification
|
|
revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
|
|
split: test
|
|
type: mteb/tweet_sentiment_extraction
|
|
metrics:
|
|
- type: accuracy
|
|
value: 76.49405772495754
|
|
- type: f1
|
|
value: 76.73610452546936
|
|
- type: f1_weighted
|
|
value: 76.14362047024868
|
|
- type: main_score
|
|
value: 76.49405772495754
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB TwentyNewsgroupsClustering
|
|
revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
|
|
split: test
|
|
type: mteb/twentynewsgroups-clustering
|
|
metrics:
|
|
- type: main_score
|
|
value: 49.554702818248735
|
|
- type: v_measure
|
|
value: 49.554702818248735
|
|
- type: v_measure_std
|
|
value: 0.9278298624304031
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB TwitterSemEval2015
|
|
revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
|
|
split: test
|
|
type: mteb/twittersemeval2015-pairclassification
|
|
metrics:
|
|
- type: cosine_accuracy
|
|
value: 85.29534481730941
|
|
- type: cosine_accuracy_threshold
|
|
value: 90.6567394733429
|
|
- type: cosine_ap
|
|
value: 71.59976408272617
|
|
- type: cosine_f1
|
|
value: 66.54452180285818
|
|
- type: cosine_f1_threshold
|
|
value: 88.94971013069153
|
|
- type: cosine_precision
|
|
value: 61.95133045258131
|
|
- type: cosine_recall
|
|
value: 71.87335092348285
|
|
- type: dot_accuracy
|
|
value: 85.29534481730941
|
|
- type: dot_accuracy_threshold
|
|
value: 90.65674543380737
|
|
- type: dot_ap
|
|
value: 71.5997871796046
|
|
- type: dot_f1
|
|
value: 66.54452180285818
|
|
- type: dot_f1_threshold
|
|
value: 88.94971013069153
|
|
- type: dot_precision
|
|
value: 61.95133045258131
|
|
- type: dot_recall
|
|
value: 71.87335092348285
|
|
- type: euclidean_accuracy
|
|
value: 85.29534481730941
|
|
- type: euclidean_accuracy_threshold
|
|
value: 43.2279109954834
|
|
- type: euclidean_ap
|
|
value: 71.59977967634174
|
|
- type: euclidean_f1
|
|
value: 66.54452180285818
|
|
- type: euclidean_f1_threshold
|
|
value: 47.01125621795654
|
|
- type: euclidean_precision
|
|
value: 61.95133045258131
|
|
- type: euclidean_recall
|
|
value: 71.87335092348285
|
|
- type: main_score
|
|
value: 71.5997871796046
|
|
- type: manhattan_accuracy
|
|
value: 85.1820945341837
|
|
- type: manhattan_accuracy_threshold
|
|
value: 1019.9851989746094
|
|
- type: manhattan_ap
|
|
value: 71.22149639016482
|
|
- type: manhattan_f1
|
|
value: 66.31834750911301
|
|
- type: manhattan_f1_threshold
|
|
value: 1109.6149444580078
|
|
- type: manhattan_precision
|
|
value: 61.46396396396396
|
|
- type: manhattan_recall
|
|
value: 72.00527704485488
|
|
- type: max_ap
|
|
value: 71.5997871796046
|
|
- type: max_f1
|
|
value: 66.54452180285818
|
|
- type: max_precision
|
|
value: 61.95133045258131
|
|
- type: max_recall
|
|
value: 72.00527704485488
|
|
- type: similarity_accuracy
|
|
value: 85.29534481730941
|
|
- type: similarity_accuracy_threshold
|
|
value: 90.6567394733429
|
|
- type: similarity_ap
|
|
value: 71.59976408272617
|
|
- type: similarity_f1
|
|
value: 66.54452180285818
|
|
- type: similarity_f1_threshold
|
|
value: 88.94971013069153
|
|
- type: similarity_precision
|
|
value: 61.95133045258131
|
|
- type: similarity_recall
|
|
value: 71.87335092348285
|
|
task:
|
|
type: PairClassification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB TwitterURLCorpus
|
|
revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
|
|
split: test
|
|
type: mteb/twitterurlcorpus-pairclassification
|
|
metrics:
|
|
- type: cosine_accuracy
|
|
value: 89.19936352699189
|
|
- type: cosine_accuracy_threshold
|
|
value: 87.72701621055603
|
|
- type: cosine_ap
|
|
value: 86.32764736710979
|
|
- type: cosine_f1
|
|
value: 78.40269966254218
|
|
- type: cosine_f1_threshold
|
|
value: 86.80565357208252
|
|
- type: cosine_precision
|
|
value: 76.41426692004093
|
|
- type: cosine_recall
|
|
value: 80.49738219895288
|
|
- type: dot_accuracy
|
|
value: 89.19936352699189
|
|
- type: dot_accuracy_threshold
|
|
value: 87.72701621055603
|
|
- type: dot_ap
|
|
value: 86.32762879051161
|
|
- type: dot_f1
|
|
value: 78.40269966254218
|
|
- type: dot_f1_threshold
|
|
value: 86.80565357208252
|
|
- type: dot_precision
|
|
value: 76.41426692004093
|
|
- type: dot_recall
|
|
value: 80.49738219895288
|
|
- type: euclidean_accuracy
|
|
value: 89.19936352699189
|
|
- type: euclidean_accuracy_threshold
|
|
value: 49.54388439655304
|
|
- type: euclidean_ap
|
|
value: 86.3276630523782
|
|
- type: euclidean_f1
|
|
value: 78.40269966254218
|
|
- type: euclidean_f1_threshold
|
|
value: 51.36992931365967
|
|
- type: euclidean_precision
|
|
value: 76.41426692004093
|
|
- type: euclidean_recall
|
|
value: 80.49738219895288
|
|
- type: main_score
|
|
value: 86.3276630523782
|
|
- type: manhattan_accuracy
|
|
value: 89.16637559669344
|
|
- type: manhattan_accuracy_threshold
|
|
value: 1150.1700401306152
|
|
- type: manhattan_ap
|
|
value: 86.28674414277404
|
|
- type: manhattan_f1
|
|
value: 78.34183768482997
|
|
- type: manhattan_f1_threshold
|
|
value: 1213.088321685791
|
|
- type: manhattan_precision
|
|
value: 75.87475651107424
|
|
- type: manhattan_recall
|
|
value: 80.97474591931014
|
|
- type: max_ap
|
|
value: 86.3276630523782
|
|
- type: max_f1
|
|
value: 78.40269966254218
|
|
- type: max_precision
|
|
value: 76.41426692004093
|
|
- type: max_recall
|
|
value: 80.97474591931014
|
|
- type: similarity_accuracy
|
|
value: 89.19936352699189
|
|
- type: similarity_accuracy_threshold
|
|
value: 87.72701621055603
|
|
- type: similarity_ap
|
|
value: 86.32764736710979
|
|
- type: similarity_f1
|
|
value: 78.40269966254218
|
|
- type: similarity_f1_threshold
|
|
value: 86.80565357208252
|
|
- type: similarity_precision
|
|
value: 76.41426692004093
|
|
- type: similarity_recall
|
|
value: 80.49738219895288
|
|
task:
|
|
type: PairClassification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB AFQMC
|
|
revision: b44c3b011063adb25877c13823db83bb193913c4
|
|
split: validation
|
|
type: C-MTEB/AFQMC
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 38.59465613249044
|
|
- type: cosine_spearman
|
|
value: 39.876884773191065
|
|
- type: euclidean_pearson
|
|
value: 38.370163017159996
|
|
- type: euclidean_spearman
|
|
value: 39.87692498028858
|
|
- type: main_score
|
|
value: 39.876884773191065
|
|
- type: manhattan_pearson
|
|
value: 38.058013850119785
|
|
- type: manhattan_spearman
|
|
value: 39.531271872106856
|
|
- type: pearson
|
|
value: 38.59465613249044
|
|
- type: spearman
|
|
value: 39.876884773191065
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB ATEC
|
|
revision: 0f319b1142f28d00e055a6770f3f726ae9b7d865
|
|
split: test
|
|
type: C-MTEB/ATEC
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 46.457799031090666
|
|
- type: cosine_spearman
|
|
value: 47.170032935367935
|
|
- type: euclidean_pearson
|
|
value: 49.399858337266004
|
|
- type: euclidean_spearman
|
|
value: 47.17003293450119
|
|
- type: main_score
|
|
value: 47.170032935367935
|
|
- type: manhattan_pearson
|
|
value: 49.19428772786887
|
|
- type: manhattan_spearman
|
|
value: 46.94649743167009
|
|
- type: pearson
|
|
value: 46.457799031090666
|
|
- type: spearman
|
|
value: 47.170032935367935
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: zh
|
|
name: MTEB AmazonReviewsClassification (zh)
|
|
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
|
|
split: test
|
|
type: mteb/amazon_reviews_multi
|
|
metrics:
|
|
- type: accuracy
|
|
value: 50.246
|
|
- type: f1
|
|
value: 45.84988588370862
|
|
- type: f1_weighted
|
|
value: 45.84988588370862
|
|
- type: main_score
|
|
value: 50.246
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB BQ
|
|
revision: e3dda5e115e487b39ec7e618c0c6a29137052a55
|
|
split: test
|
|
type: C-MTEB/BQ
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 53.67950003884396
|
|
- type: cosine_spearman
|
|
value: 54.36088598761955
|
|
- type: euclidean_pearson
|
|
value: 53.09394654913335
|
|
- type: euclidean_spearman
|
|
value: 54.36088252221325
|
|
- type: main_score
|
|
value: 54.36088598761955
|
|
- type: manhattan_pearson
|
|
value: 52.805415867146955
|
|
- type: manhattan_spearman
|
|
value: 54.06705049402532
|
|
- type: pearson
|
|
value: 53.67950003884396
|
|
- type: spearman
|
|
value: 54.36088598761955
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CLSClusteringP2P
|
|
revision: 4b6227591c6c1a73bc76b1055f3b7f3588e72476
|
|
split: test
|
|
type: C-MTEB/CLSClusteringP2P
|
|
metrics:
|
|
- type: main_score
|
|
value: 41.608876653105966
|
|
- type: v_measure
|
|
value: 41.608876653105966
|
|
- type: v_measure_std
|
|
value: 1.0624705258546963
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CLSClusteringS2S
|
|
revision: e458b3f5414b62b7f9f83499ac1f5497ae2e869f
|
|
split: test
|
|
type: C-MTEB/CLSClusteringS2S
|
|
metrics:
|
|
- type: main_score
|
|
value: 39.7110966049789
|
|
- type: v_measure
|
|
value: 39.7110966049789
|
|
- type: v_measure_std
|
|
value: 0.875231943450341
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CMedQAv1
|
|
revision: 8d7f1e942507dac42dc58017c1a001c3717da7df
|
|
split: test
|
|
type: C-MTEB/CMedQAv1-reranking
|
|
metrics:
|
|
- type: main_score
|
|
value: 81.7193302624052
|
|
- type: map
|
|
value: 81.7193302624052
|
|
- type: mrr
|
|
value: 84.58841269841271
|
|
- type: nAUC_map_diff1
|
|
value: 57.41916975321788
|
|
- type: nAUC_map_max
|
|
value: 61.409376634272874
|
|
- type: nAUC_map_std
|
|
value: 28.913154318201233
|
|
- type: nAUC_mrr_diff1
|
|
value: 64.85350793018186
|
|
- type: nAUC_mrr_max
|
|
value: 69.46338529223004
|
|
- type: nAUC_mrr_std
|
|
value: 35.373588518165235
|
|
task:
|
|
type: Reranking
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CMedQAv2
|
|
revision: 23d186750531a14a0357ca22cd92d712fd512ea0
|
|
split: test
|
|
type: C-MTEB/CMedQAv2-reranking
|
|
metrics:
|
|
- type: main_score
|
|
value: 82.59163356780259
|
|
- type: map
|
|
value: 82.59163356780259
|
|
- type: mrr
|
|
value: 85.54900793650792
|
|
- type: nAUC_map_diff1
|
|
value: 61.10665055831455
|
|
- type: nAUC_map_max
|
|
value: 60.91441391850925
|
|
- type: nAUC_map_std
|
|
value: 21.471788062972436
|
|
- type: nAUC_mrr_diff1
|
|
value: 69.95883630916767
|
|
- type: nAUC_mrr_max
|
|
value: 71.06959737866757
|
|
- type: nAUC_mrr_std
|
|
value: 30.819473605657606
|
|
task:
|
|
type: Reranking
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CmedqaRetrieval
|
|
revision: cd540c506dae1cf9e9a59c3e06f42030d54e7301
|
|
split: dev
|
|
type: C-MTEB/CmedqaRetrieval
|
|
metrics:
|
|
- type: main_score
|
|
value: 42.631
|
|
- type: map_at_1
|
|
value: 24.834
|
|
- type: map_at_10
|
|
value: 36.447
|
|
- type: map_at_100
|
|
value: 38.04
|
|
- type: map_at_1000
|
|
value: 38.179
|
|
- type: map_at_20
|
|
value: 37.281
|
|
- type: map_at_3
|
|
value: 32.761
|
|
- type: map_at_5
|
|
value: 34.871
|
|
- type: mrr_at_1
|
|
value: 38.05951487871968
|
|
- type: mrr_at_10
|
|
value: 45.57554071057435
|
|
- type: mrr_at_100
|
|
value: 46.447190120013
|
|
- type: mrr_at_1000
|
|
value: 46.50606585607273
|
|
- type: mrr_at_20
|
|
value: 46.057122452003696
|
|
- type: mrr_at_3
|
|
value: 43.34000166708336
|
|
- type: mrr_at_5
|
|
value: 44.58531299491537
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 48.47252945149055
|
|
- type: nauc_map_at_1000_max
|
|
value: 34.62100533042246
|
|
- type: nauc_map_at_1000_std
|
|
value: -2.684326419049642
|
|
- type: nauc_map_at_100_diff1
|
|
value: 48.43175156549248
|
|
- type: nauc_map_at_100_max
|
|
value: 34.58371253483366
|
|
- type: nauc_map_at_100_std
|
|
value: -2.719072576245476
|
|
- type: nauc_map_at_10_diff1
|
|
value: 48.18476956739444
|
|
- type: nauc_map_at_10_max
|
|
value: 33.52918292302435
|
|
- type: nauc_map_at_10_std
|
|
value: -3.746440821126843
|
|
- type: nauc_map_at_1_diff1
|
|
value: 52.68253139221022
|
|
- type: nauc_map_at_1_max
|
|
value: 26.033202075590157
|
|
- type: nauc_map_at_1_std
|
|
value: -5.756330655143574
|
|
- type: nauc_map_at_20_diff1
|
|
value: 48.33335064427594
|
|
- type: nauc_map_at_20_max
|
|
value: 34.08423189594616
|
|
- type: nauc_map_at_20_std
|
|
value: -3.2957587803371693
|
|
- type: nauc_map_at_3_diff1
|
|
value: 49.07970552101722
|
|
- type: nauc_map_at_3_max
|
|
value: 30.931354812941592
|
|
- type: nauc_map_at_3_std
|
|
value: -5.397714078300849
|
|
- type: nauc_map_at_5_diff1
|
|
value: 48.582852045037974
|
|
- type: nauc_map_at_5_max
|
|
value: 32.37350218464533
|
|
- type: nauc_map_at_5_std
|
|
value: -4.604286079722004
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 55.36516647246729
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 41.8197309169163
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 1.2938880389263046
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 55.33480230365865
|
|
- type: nauc_mrr_at_100_max
|
|
value: 41.82044267368069
|
|
- type: nauc_mrr_at_100_std
|
|
value: 1.3168989639934452
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 55.25761484350501
|
|
- type: nauc_mrr_at_10_max
|
|
value: 41.625145381930565
|
|
- type: nauc_mrr_at_10_std
|
|
value: 1.0129282219497187
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 60.68654871568434
|
|
- type: nauc_mrr_at_1_max
|
|
value: 43.033167419208546
|
|
- type: nauc_mrr_at_1_std
|
|
value: 0.4003726817671297
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 55.265505678078995
|
|
- type: nauc_mrr_at_20_max
|
|
value: 41.7232926738926
|
|
- type: nauc_mrr_at_20_std
|
|
value: 1.1959474260609984
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 56.49797535079964
|
|
- type: nauc_mrr_at_3_max
|
|
value: 41.922468081636865
|
|
- type: nauc_mrr_at_3_std
|
|
value: 0.7461678066019137
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 55.726696029505305
|
|
- type: nauc_mrr_at_5_max
|
|
value: 41.7068087576993
|
|
- type: nauc_mrr_at_5_std
|
|
value: 0.9345604936396126
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 49.12475845061519
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 38.13450613159849
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 0.9070870161011241
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 48.12044160559342
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 37.98858612073559
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 1.398027778560473
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 47.49083707975477
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 35.424124038022484
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -1.9285006153671742
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 60.68654871568434
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 43.033167419208546
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 0.4003726817671297
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 47.692259910508014
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 36.20333827999666
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -1.1366081258269927
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 49.926059859304004
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 36.554915901584614
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -1.7727717324767251
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 48.504726113001304
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 35.2222520201459
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -2.1147823162180046
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 5.95771915067704
|
|
- type: nauc_precision_at_1000_max
|
|
value: 29.222734901088483
|
|
- type: nauc_precision_at_1000_std
|
|
value: 21.021319062045514
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 12.441767269549631
|
|
- type: nauc_precision_at_100_max
|
|
value: 37.028610753731876
|
|
- type: nauc_precision_at_100_std
|
|
value: 22.59370573792191
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 25.3055255305395
|
|
- type: nauc_precision_at_10_max
|
|
value: 41.57346735024518
|
|
- type: nauc_precision_at_10_std
|
|
value: 10.851514810119529
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 60.68654871568434
|
|
- type: nauc_precision_at_1_max
|
|
value: 43.033167419208546
|
|
- type: nauc_precision_at_1_std
|
|
value: 0.4003726817671297
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 21.503387725334118
|
|
- type: nauc_precision_at_20_max
|
|
value: 40.35637914704234
|
|
- type: nauc_precision_at_20_std
|
|
value: 14.15622720179941
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 37.92102588120911
|
|
- type: nauc_precision_at_3_max
|
|
value: 42.61959379379323
|
|
- type: nauc_precision_at_3_std
|
|
value: 4.531204029823331
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 31.822114101121624
|
|
- type: nauc_precision_at_5_max
|
|
value: 42.00621213856077
|
|
- type: nauc_precision_at_5_std
|
|
value: 7.038453918682581
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 30.906717381989445
|
|
- type: nauc_recall_at_1000_max
|
|
value: 49.86631344507457
|
|
- type: nauc_recall_at_1000_std
|
|
value: 44.77133994051694
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 29.06337979940958
|
|
- type: nauc_recall_at_100_max
|
|
value: 35.64030149194558
|
|
- type: nauc_recall_at_100_std
|
|
value: 16.019430611168264
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 34.92848768468913
|
|
- type: nauc_recall_at_10_max
|
|
value: 28.566945065867454
|
|
- type: nauc_recall_at_10_std
|
|
value: -2.1058035354561557
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 52.68253139221022
|
|
- type: nauc_recall_at_1_max
|
|
value: 26.033202075590157
|
|
- type: nauc_recall_at_1_std
|
|
value: -5.756330655143574
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 33.82932775397309
|
|
- type: nauc_recall_at_20_max
|
|
value: 29.679872190739044
|
|
- type: nauc_recall_at_20_std
|
|
value: 0.10165951410954753
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 42.53700938223526
|
|
- type: nauc_recall_at_3_max
|
|
value: 27.477725171266385
|
|
- type: nauc_recall_at_3_std
|
|
value: -5.201557627828334
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 39.158896850349116
|
|
- type: nauc_recall_at_5_max
|
|
value: 27.90842581577196
|
|
- type: nauc_recall_at_5_std
|
|
value: -3.646479982111823
|
|
- type: ndcg_at_1
|
|
value: 38.06
|
|
- type: ndcg_at_10
|
|
value: 42.631
|
|
- type: ndcg_at_100
|
|
value: 49.114000000000004
|
|
- type: ndcg_at_1000
|
|
value: 51.745
|
|
- type: ndcg_at_20
|
|
value: 44.895
|
|
- type: ndcg_at_3
|
|
value: 38.153999999999996
|
|
- type: ndcg_at_5
|
|
value: 39.994
|
|
- type: precision_at_1
|
|
value: 38.06
|
|
- type: precision_at_10
|
|
value: 9.35
|
|
- type: precision_at_100
|
|
value: 1.471
|
|
- type: precision_at_1000
|
|
value: 0.181
|
|
- type: precision_at_20
|
|
value: 5.461
|
|
- type: precision_at_3
|
|
value: 21.555
|
|
- type: precision_at_5
|
|
value: 15.443999999999999
|
|
- type: recall_at_1
|
|
value: 24.834
|
|
- type: recall_at_10
|
|
value: 51.881
|
|
- type: recall_at_100
|
|
value: 79.095
|
|
- type: recall_at_1000
|
|
value: 97.077
|
|
- type: recall_at_20
|
|
value: 59.471
|
|
- type: recall_at_3
|
|
value: 37.836
|
|
- type: recall_at_5
|
|
value: 43.913999999999994
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB Cmnli
|
|
revision: 41bc36f332156f7adc9e38f53777c959b2ae9766
|
|
split: validation
|
|
type: C-MTEB/CMNLI
|
|
metrics:
|
|
- type: cosine_accuracy
|
|
value: 69.29645219482863
|
|
- type: cosine_accuracy_threshold
|
|
value: 83.89029502868652
|
|
- type: cosine_ap
|
|
value: 76.28529631089978
|
|
- type: cosine_f1
|
|
value: 72.18316549496485
|
|
- type: cosine_f1_threshold
|
|
value: 79.37869429588318
|
|
- type: cosine_precision
|
|
value: 60.79372699631941
|
|
- type: cosine_recall
|
|
value: 88.82394201543138
|
|
- type: dot_accuracy
|
|
value: 69.29645219482863
|
|
- type: dot_accuracy_threshold
|
|
value: 83.890300989151
|
|
- type: dot_ap
|
|
value: 76.28533525182606
|
|
- type: dot_f1
|
|
value: 72.18316549496485
|
|
- type: dot_f1_threshold
|
|
value: 79.37869429588318
|
|
- type: dot_precision
|
|
value: 60.79372699631941
|
|
- type: dot_recall
|
|
value: 88.82394201543138
|
|
- type: euclidean_accuracy
|
|
value: 69.29645219482863
|
|
- type: euclidean_accuracy_threshold
|
|
value: 56.762146949768066
|
|
- type: euclidean_ap
|
|
value: 76.28547969937172
|
|
- type: euclidean_f1
|
|
value: 72.18316549496485
|
|
- type: euclidean_f1_threshold
|
|
value: 64.22040462493896
|
|
- type: euclidean_precision
|
|
value: 60.79372699631941
|
|
- type: euclidean_recall
|
|
value: 88.82394201543138
|
|
- type: main_score
|
|
value: 76.28547969937172
|
|
- type: manhattan_accuracy
|
|
value: 68.86349969933855
|
|
- type: manhattan_accuracy_threshold
|
|
value: 1325.539207458496
|
|
- type: manhattan_ap
|
|
value: 75.73527179489312
|
|
- type: manhattan_f1
|
|
value: 71.93284110448064
|
|
- type: manhattan_f1_threshold
|
|
value: 1450.2345085144043
|
|
- type: manhattan_precision
|
|
value: 63.386809269162214
|
|
- type: manhattan_recall
|
|
value: 83.14238952536824
|
|
- type: max_ap
|
|
value: 76.28547969937172
|
|
- type: max_f1
|
|
value: 72.18316549496485
|
|
- type: max_precision
|
|
value: 63.386809269162214
|
|
- type: max_recall
|
|
value: 88.82394201543138
|
|
- type: similarity_accuracy
|
|
value: 69.29645219482863
|
|
- type: similarity_accuracy_threshold
|
|
value: 83.89029502868652
|
|
- type: similarity_ap
|
|
value: 76.28529631089978
|
|
- type: similarity_f1
|
|
value: 72.18316549496485
|
|
- type: similarity_f1_threshold
|
|
value: 79.37869429588318
|
|
- type: similarity_precision
|
|
value: 60.79372699631941
|
|
- type: similarity_recall
|
|
value: 88.82394201543138
|
|
task:
|
|
type: PairClassification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CovidRetrieval
|
|
revision: 1271c7809071a13532e05f25fb53511ffce77117
|
|
split: dev
|
|
type: C-MTEB/CovidRetrieval
|
|
metrics:
|
|
- type: main_score
|
|
value: 82.76599999999999
|
|
- type: map_at_1
|
|
value: 70.99600000000001
|
|
- type: map_at_10
|
|
value: 79.022
|
|
- type: map_at_100
|
|
value: 79.262
|
|
- type: map_at_1000
|
|
value: 79.266
|
|
- type: map_at_20
|
|
value: 79.211
|
|
- type: map_at_3
|
|
value: 77.081
|
|
- type: map_at_5
|
|
value: 78.348
|
|
- type: mrr_at_1
|
|
value: 71.12750263435194
|
|
- type: mrr_at_10
|
|
value: 79.00563667686959
|
|
- type: mrr_at_100
|
|
value: 79.24545000482046
|
|
- type: mrr_at_1000
|
|
value: 79.24986213861123
|
|
- type: mrr_at_20
|
|
value: 79.19503716749968
|
|
- type: mrr_at_3
|
|
value: 77.1338250790306
|
|
- type: mrr_at_5
|
|
value: 78.38250790305591
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 79.78007097062118
|
|
- type: nauc_map_at_1000_max
|
|
value: 31.495494389521216
|
|
- type: nauc_map_at_1000_std
|
|
value: -44.554113523471585
|
|
- type: nauc_map_at_100_diff1
|
|
value: 79.77901003479913
|
|
- type: nauc_map_at_100_max
|
|
value: 31.501728637681925
|
|
- type: nauc_map_at_100_std
|
|
value: -44.54526589087225
|
|
- type: nauc_map_at_10_diff1
|
|
value: 79.70465086616332
|
|
- type: nauc_map_at_10_max
|
|
value: 31.447942385382856
|
|
- type: nauc_map_at_10_std
|
|
value: -44.86102015819248
|
|
- type: nauc_map_at_1_diff1
|
|
value: 81.89774804895447
|
|
- type: nauc_map_at_1_max
|
|
value: 29.53109235427305
|
|
- type: nauc_map_at_1_std
|
|
value: -42.80277721451948
|
|
- type: nauc_map_at_20_diff1
|
|
value: 79.77871635635559
|
|
- type: nauc_map_at_20_max
|
|
value: 31.560274527206733
|
|
- type: nauc_map_at_20_std
|
|
value: -44.55008236120152
|
|
- type: nauc_map_at_3_diff1
|
|
value: 79.37871528079008
|
|
- type: nauc_map_at_3_max
|
|
value: 30.314627717947655
|
|
- type: nauc_map_at_3_std
|
|
value: -46.583081505018214
|
|
- type: nauc_map_at_5_diff1
|
|
value: 79.47410569600237
|
|
- type: nauc_map_at_5_max
|
|
value: 30.717452787943255
|
|
- type: nauc_map_at_5_std
|
|
value: -45.56487302807213
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 79.83396133475738
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 31.902081193300802
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -44.32825329012893
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 79.832888351025
|
|
- type: nauc_mrr_at_100_max
|
|
value: 31.90821451879506
|
|
- type: nauc_mrr_at_100_std
|
|
value: -44.31946551133598
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 79.75766328526763
|
|
- type: nauc_mrr_at_10_max
|
|
value: 31.84709271229474
|
|
- type: nauc_mrr_at_10_std
|
|
value: -44.64251370779262
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 81.88675621341875
|
|
- type: nauc_mrr_at_1_max
|
|
value: 30.624768062722435
|
|
- type: nauc_mrr_at_1_std
|
|
value: -41.826968180693456
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 79.83221800317402
|
|
- type: nauc_mrr_at_20_max
|
|
value: 31.96340672339527
|
|
- type: nauc_mrr_at_20_std
|
|
value: -44.32956320098315
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 79.34629346809106
|
|
- type: nauc_mrr_at_3_max
|
|
value: 31.358295528236113
|
|
- type: nauc_mrr_at_3_std
|
|
value: -45.97803582281396
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 79.494177213373
|
|
- type: nauc_mrr_at_5_max
|
|
value: 31.52236804483443
|
|
- type: nauc_mrr_at_5_std
|
|
value: -45.138775893398694
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 79.42223230573576
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 32.28843903409106
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -44.3133954110294
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 79.3929907054809
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 32.49291426150998
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -43.996604718501075
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 79.11644773352661
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 32.54744027915217
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -45.44820798746672
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 81.71471193659804
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 30.56723762753589
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -42.00582595178881
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 79.34070754205227
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 33.08175655505984
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -43.93297429354463
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 78.41040890432154
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 30.540602587995053
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -48.682741281966244
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 78.52045059102817
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 31.145620595701786
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -46.96161213475506
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -26.20700295711843
|
|
- type: nauc_precision_at_1000_max
|
|
value: 50.992072309587066
|
|
- type: nauc_precision_at_1000_std
|
|
value: 49.034232966809896
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -1.2318650992746658
|
|
- type: nauc_precision_at_100_max
|
|
value: 54.103623972545876
|
|
- type: nauc_precision_at_100_std
|
|
value: 38.158651434354105
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 47.40081635911143
|
|
- type: nauc_precision_at_10_max
|
|
value: 46.01760789553407
|
|
- type: nauc_precision_at_10_std
|
|
value: -22.545587533051467
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 81.71471193659804
|
|
- type: nauc_precision_at_1_max
|
|
value: 30.56723762753589
|
|
- type: nauc_precision_at_1_std
|
|
value: -42.00582595178881
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 31.902645462266044
|
|
- type: nauc_precision_at_20_max
|
|
value: 60.06037928799191
|
|
- type: nauc_precision_at_20_std
|
|
value: 10.125381568485691
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 70.23181696295782
|
|
- type: nauc_precision_at_3_max
|
|
value: 31.33307476962615
|
|
- type: nauc_precision_at_3_std
|
|
value: -52.773523783308995
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 63.24118340779976
|
|
- type: nauc_precision_at_5_max
|
|
value: 35.536460706118284
|
|
- type: nauc_precision_at_5_std
|
|
value: -43.859100503715496
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 63.10783066308766
|
|
- type: nauc_recall_at_1000_max
|
|
value: 64.17746555050037
|
|
- type: nauc_recall_at_1000_std
|
|
value: -1.1314627694685895
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 70.70747402244945
|
|
- type: nauc_recall_at_100_max
|
|
value: 63.81462634298472
|
|
- type: nauc_recall_at_100_std
|
|
value: 2.7329437124855858
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 74.5724683430861
|
|
- type: nauc_recall_at_10_max
|
|
value: 42.06028697147503
|
|
- type: nauc_recall_at_10_std
|
|
value: -50.426163431789384
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 81.89774804895447
|
|
- type: nauc_recall_at_1_max
|
|
value: 29.53109235427305
|
|
- type: nauc_recall_at_1_std
|
|
value: -42.80277721451948
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 74.1386367152198
|
|
- type: nauc_recall_at_20_max
|
|
value: 60.26605112943992
|
|
- type: nauc_recall_at_20_std
|
|
value: -24.167905489617926
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 74.68360442418249
|
|
- type: nauc_recall_at_3_max
|
|
value: 29.73174978017023
|
|
- type: nauc_recall_at_3_std
|
|
value: -58.048521143234844
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 73.33434605574439
|
|
- type: nauc_recall_at_5_max
|
|
value: 31.829043506426963
|
|
- type: nauc_recall_at_5_std
|
|
value: -55.33176739081927
|
|
- type: ndcg_at_1
|
|
value: 71.233
|
|
- type: ndcg_at_10
|
|
value: 82.76599999999999
|
|
- type: ndcg_at_100
|
|
value: 83.799
|
|
- type: ndcg_at_1000
|
|
value: 83.898
|
|
- type: ndcg_at_20
|
|
value: 83.44
|
|
- type: ndcg_at_3
|
|
value: 79.03999999999999
|
|
- type: ndcg_at_5
|
|
value: 81.285
|
|
- type: precision_at_1
|
|
value: 71.233
|
|
- type: precision_at_10
|
|
value: 9.526
|
|
- type: precision_at_100
|
|
value: 0.9990000000000001
|
|
- type: precision_at_1000
|
|
value: 0.101
|
|
- type: precision_at_20
|
|
value: 4.8950000000000005
|
|
- type: precision_at_3
|
|
value: 28.346
|
|
- type: precision_at_5
|
|
value: 18.124000000000002
|
|
- type: recall_at_1
|
|
value: 70.99600000000001
|
|
- type: recall_at_10
|
|
value: 94.31
|
|
- type: recall_at_100
|
|
value: 98.84100000000001
|
|
- type: recall_at_1000
|
|
value: 99.579
|
|
- type: recall_at_20
|
|
value: 96.944
|
|
- type: recall_at_3
|
|
value: 84.589
|
|
- type: recall_at_5
|
|
value: 89.98899999999999
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB DuRetrieval
|
|
revision: a1a333e290fe30b10f3f56498e3a0d911a693ced
|
|
split: dev
|
|
type: C-MTEB/DuRetrieval
|
|
metrics:
|
|
- type: main_score
|
|
value: 82.353
|
|
- type: map_at_1
|
|
value: 23.408
|
|
- type: map_at_10
|
|
value: 73.302
|
|
- type: map_at_100
|
|
value: 76.532
|
|
- type: map_at_1000
|
|
value: 76.578
|
|
- type: map_at_20
|
|
value: 75.765
|
|
- type: map_at_3
|
|
value: 49.297999999999995
|
|
- type: map_at_5
|
|
value: 62.96000000000001
|
|
- type: mrr_at_1
|
|
value: 83.3
|
|
- type: mrr_at_10
|
|
value: 88.85841269841264
|
|
- type: mrr_at_100
|
|
value: 88.937851229216
|
|
- type: mrr_at_1000
|
|
value: 88.94253811030754
|
|
- type: mrr_at_20
|
|
value: 88.90522789194803
|
|
- type: mrr_at_3
|
|
value: 88.31666666666662
|
|
- type: mrr_at_5
|
|
value: 88.66416666666662
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 3.978108165433077
|
|
- type: nauc_map_at_1000_max
|
|
value: 32.84013060265069
|
|
- type: nauc_map_at_1000_std
|
|
value: 17.104374545928255
|
|
- type: nauc_map_at_100_diff1
|
|
value: 3.9594456007844183
|
|
- type: nauc_map_at_100_max
|
|
value: 32.84323698444807
|
|
- type: nauc_map_at_100_std
|
|
value: 17.083360165851175
|
|
- type: nauc_map_at_10_diff1
|
|
value: 6.564428602685249
|
|
- type: nauc_map_at_10_max
|
|
value: 29.490007273766956
|
|
- type: nauc_map_at_10_std
|
|
value: 6.955854455105477
|
|
- type: nauc_map_at_1_diff1
|
|
value: 43.01902060700144
|
|
- type: nauc_map_at_1_max
|
|
value: -8.940094269879843
|
|
- type: nauc_map_at_1_std
|
|
value: -28.063233795166276
|
|
- type: nauc_map_at_20_diff1
|
|
value: 4.446904145850981
|
|
- type: nauc_map_at_20_max
|
|
value: 32.47424290913474
|
|
- type: nauc_map_at_20_std
|
|
value: 14.957146942696257
|
|
- type: nauc_map_at_3_diff1
|
|
value: 25.91745605988797
|
|
- type: nauc_map_at_3_max
|
|
value: 3.661124903759869
|
|
- type: nauc_map_at_3_std
|
|
value: -21.936610233451646
|
|
- type: nauc_map_at_5_diff1
|
|
value: 16.629939273347865
|
|
- type: nauc_map_at_5_max
|
|
value: 14.666913498454564
|
|
- type: nauc_map_at_5_std
|
|
value: -12.39941441022446
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 26.08525262735903
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 47.86393129438558
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 28.811634091001743
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 26.081836904532153
|
|
- type: nauc_mrr_at_100_max
|
|
value: 47.880134050815
|
|
- type: nauc_mrr_at_100_std
|
|
value: 28.828980969011475
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 26.09549377249783
|
|
- type: nauc_mrr_at_10_max
|
|
value: 48.11004429436051
|
|
- type: nauc_mrr_at_10_std
|
|
value: 29.041772733561455
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 26.095576390896717
|
|
- type: nauc_mrr_at_1_max
|
|
value: 40.102786808829485
|
|
- type: nauc_mrr_at_1_std
|
|
value: 21.16142603421125
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 26.078553311053394
|
|
- type: nauc_mrr_at_20_max
|
|
value: 47.9955055491724
|
|
- type: nauc_mrr_at_20_std
|
|
value: 28.92844826033336
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 25.736821420614447
|
|
- type: nauc_mrr_at_3_max
|
|
value: 48.30695057366758
|
|
- type: nauc_mrr_at_3_std
|
|
value: 29.295726311215475
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 25.979034861669714
|
|
- type: nauc_mrr_at_5_max
|
|
value: 48.500915285456344
|
|
- type: nauc_mrr_at_5_std
|
|
value: 29.449704923164106
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 6.624272455812551
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 41.526519286613414
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 27.91983541845217
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 6.033169661320914
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 41.6841728152419
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 28.35967524719135
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 5.627968389448389
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 37.18261001317417
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 19.757054878692408
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 26.095576390896717
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 40.102786808829485
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 21.16142603421125
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 5.678380464964442
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 40.70268508824627
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 25.003203457508622
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 5.7196343030730645
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 34.50950904905902
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 20.099411226966403
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 7.398974214665505
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 31.777872881596885
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 14.212532410116573
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -26.784369186388286
|
|
- type: nauc_precision_at_1000_max
|
|
value: 20.9055343942668
|
|
- type: nauc_precision_at_1000_std
|
|
value: 48.97851074406537
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -27.79381730090699
|
|
- type: nauc_precision_at_100_max
|
|
value: 22.80005440633608
|
|
- type: nauc_precision_at_100_std
|
|
value: 50.935594672026795
|
|
- type: nauc_precision_at_10_diff1
|
|
value: -30.285772529280557
|
|
- type: nauc_precision_at_10_max
|
|
value: 32.73392928068347
|
|
- type: nauc_precision_at_10_std
|
|
value: 47.96878369413408
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 26.095576390896717
|
|
- type: nauc_precision_at_1_max
|
|
value: 40.102786808829485
|
|
- type: nauc_precision_at_1_std
|
|
value: 21.16142603421125
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -28.93118180068221
|
|
- type: nauc_precision_at_20_max
|
|
value: 27.34554979821627
|
|
- type: nauc_precision_at_20_std
|
|
value: 50.768062841591245
|
|
- type: nauc_precision_at_3_diff1
|
|
value: -20.842604987632818
|
|
- type: nauc_precision_at_3_max
|
|
value: 38.567385349160865
|
|
- type: nauc_precision_at_3_std
|
|
value: 34.962189381111585
|
|
- type: nauc_precision_at_5_diff1
|
|
value: -27.39434681486595
|
|
- type: nauc_precision_at_5_max
|
|
value: 36.46059763518038
|
|
- type: nauc_precision_at_5_std
|
|
value: 39.893251684847286
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: -11.949093496228018
|
|
- type: nauc_recall_at_1000_max
|
|
value: 73.88534051191724
|
|
- type: nauc_recall_at_1000_std
|
|
value: 74.63173870654316
|
|
- type: nauc_recall_at_100_diff1
|
|
value: -10.612653444299633
|
|
- type: nauc_recall_at_100_max
|
|
value: 55.332461824335255
|
|
- type: nauc_recall_at_100_std
|
|
value: 55.6971441098854
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 1.6381390695279527
|
|
- type: nauc_recall_at_10_max
|
|
value: 30.7773121587242
|
|
- type: nauc_recall_at_10_std
|
|
value: 5.983376763709044
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 43.01902060700144
|
|
- type: nauc_recall_at_1_max
|
|
value: -8.940094269879843
|
|
- type: nauc_recall_at_1_std
|
|
value: -28.063233795166276
|
|
- type: nauc_recall_at_20_diff1
|
|
value: -3.5879888483690268
|
|
- type: nauc_recall_at_20_max
|
|
value: 42.56780359254684
|
|
- type: nauc_recall_at_20_std
|
|
value: 28.64620011473346
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 24.423753178927363
|
|
- type: nauc_recall_at_3_max
|
|
value: 0.28631207577281326
|
|
- type: nauc_recall_at_3_std
|
|
value: -24.79099042560129
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 15.716357450134492
|
|
- type: nauc_recall_at_5_max
|
|
value: 9.923967009889193
|
|
- type: nauc_recall_at_5_std
|
|
value: -18.11714448988651
|
|
- type: ndcg_at_1
|
|
value: 83.3
|
|
- type: ndcg_at_10
|
|
value: 82.353
|
|
- type: ndcg_at_100
|
|
value: 85.952
|
|
- type: ndcg_at_1000
|
|
value: 86.393
|
|
- type: ndcg_at_20
|
|
value: 84.333
|
|
- type: ndcg_at_3
|
|
value: 79.128
|
|
- type: ndcg_at_5
|
|
value: 78.96300000000001
|
|
- type: precision_at_1
|
|
value: 83.3
|
|
- type: precision_at_10
|
|
value: 40.36
|
|
- type: precision_at_100
|
|
value: 4.769
|
|
- type: precision_at_1000
|
|
value: 0.488
|
|
- type: precision_at_20
|
|
value: 22.295
|
|
- type: precision_at_3
|
|
value: 71.25
|
|
- type: precision_at_5
|
|
value: 61.18
|
|
- type: recall_at_1
|
|
value: 23.408
|
|
- type: recall_at_10
|
|
value: 85.44800000000001
|
|
- type: recall_at_100
|
|
value: 96.712
|
|
- type: recall_at_1000
|
|
value: 98.988
|
|
- type: recall_at_20
|
|
value: 91.304
|
|
- type: recall_at_3
|
|
value: 52.65
|
|
- type: recall_at_5
|
|
value: 69.81
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB EcomRetrieval
|
|
revision: 687de13dc7294d6fd9be10c6945f9e8fec8166b9
|
|
split: dev
|
|
type: C-MTEB/EcomRetrieval
|
|
metrics:
|
|
- type: main_score
|
|
value: 63.712999999999994
|
|
- type: map_at_1
|
|
value: 49.0
|
|
- type: map_at_10
|
|
value: 58.620000000000005
|
|
- type: map_at_100
|
|
value: 59.183
|
|
- type: map_at_1000
|
|
value: 59.19799999999999
|
|
- type: map_at_20
|
|
value: 58.948
|
|
- type: map_at_3
|
|
value: 55.883
|
|
- type: map_at_5
|
|
value: 57.452999999999996
|
|
- type: mrr_at_1
|
|
value: 49.0
|
|
- type: mrr_at_10
|
|
value: 58.61988095238089
|
|
- type: mrr_at_100
|
|
value: 59.18251462760907
|
|
- type: mrr_at_1000
|
|
value: 59.1981896580556
|
|
- type: mrr_at_20
|
|
value: 58.94805232134562
|
|
- type: mrr_at_3
|
|
value: 55.883333333333304
|
|
- type: mrr_at_5
|
|
value: 57.4533333333333
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 60.33101842801658
|
|
- type: nauc_map_at_1000_max
|
|
value: 19.502683068762945
|
|
- type: nauc_map_at_1000_std
|
|
value: -9.052741690420172
|
|
- type: nauc_map_at_100_diff1
|
|
value: 60.320202163437884
|
|
- type: nauc_map_at_100_max
|
|
value: 19.511425958183473
|
|
- type: nauc_map_at_100_std
|
|
value: -9.046775711361885
|
|
- type: nauc_map_at_10_diff1
|
|
value: 60.32228179956949
|
|
- type: nauc_map_at_10_max
|
|
value: 19.6159978656515
|
|
- type: nauc_map_at_10_std
|
|
value: -9.132522477977544
|
|
- type: nauc_map_at_1_diff1
|
|
value: 61.89621977613427
|
|
- type: nauc_map_at_1_max
|
|
value: 15.015734335373715
|
|
- type: nauc_map_at_1_std
|
|
value: -12.641774992365185
|
|
- type: nauc_map_at_20_diff1
|
|
value: 60.351130642660486
|
|
- type: nauc_map_at_20_max
|
|
value: 19.433343357030232
|
|
- type: nauc_map_at_20_std
|
|
value: -9.21598413872683
|
|
- type: nauc_map_at_3_diff1
|
|
value: 60.26725821298107
|
|
- type: nauc_map_at_3_max
|
|
value: 18.3498595109406
|
|
- type: nauc_map_at_3_std
|
|
value: -10.051517839346984
|
|
- type: nauc_map_at_5_diff1
|
|
value: 60.164921439673925
|
|
- type: nauc_map_at_5_max
|
|
value: 18.593900545400267
|
|
- type: nauc_map_at_5_std
|
|
value: -9.934110598947624
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 60.33101842801658
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 19.502683068762945
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -9.052741690420172
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 60.320202163437884
|
|
- type: nauc_mrr_at_100_max
|
|
value: 19.511425958183473
|
|
- type: nauc_mrr_at_100_std
|
|
value: -9.046775711361885
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 60.32228179956949
|
|
- type: nauc_mrr_at_10_max
|
|
value: 19.6159978656515
|
|
- type: nauc_mrr_at_10_std
|
|
value: -9.132522477977544
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 61.89621977613427
|
|
- type: nauc_mrr_at_1_max
|
|
value: 15.015734335373715
|
|
- type: nauc_mrr_at_1_std
|
|
value: -12.641774992365185
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 60.351130642660486
|
|
- type: nauc_mrr_at_20_max
|
|
value: 19.433343357030232
|
|
- type: nauc_mrr_at_20_std
|
|
value: -9.21598413872683
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 60.26725821298107
|
|
- type: nauc_mrr_at_3_max
|
|
value: 18.3498595109406
|
|
- type: nauc_mrr_at_3_std
|
|
value: -10.051517839346984
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 60.164921439673925
|
|
- type: nauc_mrr_at_5_max
|
|
value: 18.593900545400267
|
|
- type: nauc_mrr_at_5_std
|
|
value: -9.934110598947624
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 60.190733838614676
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 22.361539210340222
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -5.745163462434749
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 59.89473232352801
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 22.68282893350434
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -5.4179387740783
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 60.07971889322107
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 22.591286648072977
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -6.68500894448089
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 61.89621977613427
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 15.015734335373715
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -12.641774992365185
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 60.182873920240475
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 21.964898434175247
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -6.906365610289816
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 59.8208566369894
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 19.388884168625417
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -9.151250601081255
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 59.599342583351955
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 19.8910854628725
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -8.885354650481215
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 63.58164887576627
|
|
- type: nauc_precision_at_1000_max
|
|
value: 92.23383046912454
|
|
- type: nauc_precision_at_1000_std
|
|
value: 87.13881949176067
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 53.73002142033278
|
|
- type: nauc_precision_at_100_max
|
|
value: 70.37128576920941
|
|
- type: nauc_precision_at_100_std
|
|
value: 55.41687263140533
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 59.41629120257138
|
|
- type: nauc_precision_at_10_max
|
|
value: 38.24957021696883
|
|
- type: nauc_precision_at_10_std
|
|
value: 6.335412380239172
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 61.89621977613427
|
|
- type: nauc_precision_at_1_max
|
|
value: 15.015734335373715
|
|
- type: nauc_precision_at_1_std
|
|
value: -12.641774992365185
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 59.95367722749617
|
|
- type: nauc_precision_at_20_max
|
|
value: 38.11970211089507
|
|
- type: nauc_precision_at_20_std
|
|
value: 8.468361991180146
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 58.418401476502524
|
|
- type: nauc_precision_at_3_max
|
|
value: 22.708479411978058
|
|
- type: nauc_precision_at_3_std
|
|
value: -6.238867799833925
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 57.54249152786323
|
|
- type: nauc_precision_at_5_max
|
|
value: 24.64947877432984
|
|
- type: nauc_precision_at_5_std
|
|
value: -5.018047100033905
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 63.581648875766604
|
|
- type: nauc_recall_at_1000_max
|
|
value: 92.23383046912458
|
|
- type: nauc_recall_at_1000_std
|
|
value: 87.13881949176098
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 53.73002142033278
|
|
- type: nauc_recall_at_100_max
|
|
value: 70.37128576920976
|
|
- type: nauc_recall_at_100_std
|
|
value: 55.41687263140555
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 59.41629120257145
|
|
- type: nauc_recall_at_10_max
|
|
value: 38.2495702169689
|
|
- type: nauc_recall_at_10_std
|
|
value: 6.335412380239176
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 61.89621977613427
|
|
- type: nauc_recall_at_1_max
|
|
value: 15.015734335373715
|
|
- type: nauc_recall_at_1_std
|
|
value: -12.641774992365185
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 59.95367722749639
|
|
- type: nauc_recall_at_20_max
|
|
value: 38.11970211089514
|
|
- type: nauc_recall_at_20_std
|
|
value: 8.468361991180268
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 58.41840147650248
|
|
- type: nauc_recall_at_3_max
|
|
value: 22.708479411978043
|
|
- type: nauc_recall_at_3_std
|
|
value: -6.238867799833981
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 57.542491527863206
|
|
- type: nauc_recall_at_5_max
|
|
value: 24.649478774330014
|
|
- type: nauc_recall_at_5_std
|
|
value: -5.018047100033782
|
|
- type: ndcg_at_1
|
|
value: 49.0
|
|
- type: ndcg_at_10
|
|
value: 63.712999999999994
|
|
- type: ndcg_at_100
|
|
value: 66.523
|
|
- type: ndcg_at_1000
|
|
value: 66.922
|
|
- type: ndcg_at_20
|
|
value: 64.904
|
|
- type: ndcg_at_3
|
|
value: 58.099000000000004
|
|
- type: ndcg_at_5
|
|
value: 60.913
|
|
- type: precision_at_1
|
|
value: 49.0
|
|
- type: precision_at_10
|
|
value: 7.99
|
|
- type: precision_at_100
|
|
value: 0.932
|
|
- type: precision_at_1000
|
|
value: 0.096
|
|
- type: precision_at_20
|
|
value: 4.2299999999999995
|
|
- type: precision_at_3
|
|
value: 21.5
|
|
- type: precision_at_5
|
|
value: 14.26
|
|
- type: recall_at_1
|
|
value: 49.0
|
|
- type: recall_at_10
|
|
value: 79.9
|
|
- type: recall_at_100
|
|
value: 93.2
|
|
- type: recall_at_1000
|
|
value: 96.3
|
|
- type: recall_at_20
|
|
value: 84.6
|
|
- type: recall_at_3
|
|
value: 64.5
|
|
- type: recall_at_5
|
|
value: 71.3
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB IFlyTek
|
|
revision: 421605374b29664c5fc098418fe20ada9bd55f8a
|
|
split: validation
|
|
type: C-MTEB/IFlyTek-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 49.188149288187766
|
|
- type: f1
|
|
value: 35.82742058478872
|
|
- type: f1_weighted
|
|
value: 46.33812923348324
|
|
- type: main_score
|
|
value: 49.188149288187766
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB JDReview
|
|
revision: b7c64bd89eb87f8ded463478346f76731f07bf8b
|
|
split: test
|
|
type: C-MTEB/JDReview-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 83.45215759849907
|
|
- type: ap
|
|
value: 49.602287249765666
|
|
- type: ap_weighted
|
|
value: 49.602287249765666
|
|
- type: f1
|
|
value: 77.84519218126933
|
|
- type: f1_weighted
|
|
value: 84.83784419250833
|
|
- type: main_score
|
|
value: 83.45215759849907
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB LCQMC
|
|
revision: 17f9b096f80380fce5ed12a9be8be7784b337daf
|
|
split: test
|
|
type: C-MTEB/LCQMC
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 66.78399631818323
|
|
- type: cosine_spearman
|
|
value: 70.38648345929874
|
|
- type: euclidean_pearson
|
|
value: 68.79036522204457
|
|
- type: euclidean_spearman
|
|
value: 70.38649454085622
|
|
- type: main_score
|
|
value: 70.38648345929874
|
|
- type: manhattan_pearson
|
|
value: 68.74927335399974
|
|
- type: manhattan_spearman
|
|
value: 70.3453886791424
|
|
- type: pearson
|
|
value: 66.78399631818323
|
|
- type: spearman
|
|
value: 70.38648345929874
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB MMarcoReranking
|
|
revision: 8e0c766dbe9e16e1d221116a3f36795fbade07f6
|
|
split: dev
|
|
type: C-MTEB/Mmarco-reranking
|
|
metrics:
|
|
- type: main_score
|
|
value: 26.991570930656568
|
|
- type: map
|
|
value: 26.991570930656568
|
|
- type: mrr
|
|
value: 25.460714285714285
|
|
- type: nAUC_map_diff1
|
|
value: 12.174277381054415
|
|
- type: nAUC_map_max
|
|
value: 5.768145859960792
|
|
- type: nAUC_map_std
|
|
value: -0.6863999286086584
|
|
- type: nAUC_mrr_diff1
|
|
value: 11.83053464449912
|
|
- type: nAUC_mrr_max
|
|
value: 4.893060023643725
|
|
- type: nAUC_mrr_std
|
|
value: -0.22755376963555723
|
|
task:
|
|
type: Reranking
|
|
- dataset:
|
|
config: default
|
|
name: MTEB MMarcoRetrieval
|
|
revision: 539bbde593d947e2a124ba72651aafc09eb33fc2
|
|
split: dev
|
|
type: C-MTEB/MMarcoRetrieval
|
|
metrics:
|
|
- type: main_score
|
|
value: 78.679
|
|
- type: map_at_1
|
|
value: 65.349
|
|
- type: map_at_10
|
|
value: 74.802
|
|
- type: map_at_100
|
|
value: 75.141
|
|
- type: map_at_1000
|
|
value: 75.151
|
|
- type: map_at_20
|
|
value: 75.03999999999999
|
|
- type: map_at_3
|
|
value: 72.831
|
|
- type: map_at_5
|
|
value: 74.09400000000001
|
|
- type: mrr_at_1
|
|
value: 67.55014326647564
|
|
- type: mrr_at_10
|
|
value: 75.31912038932084
|
|
- type: mrr_at_100
|
|
value: 75.6225574951573
|
|
- type: mrr_at_1000
|
|
value: 75.63176308010398
|
|
- type: mrr_at_20
|
|
value: 75.53574557856176
|
|
- type: mrr_at_3
|
|
value: 73.59598853868198
|
|
- type: mrr_at_5
|
|
value: 74.70343839541526
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 77.81972509758704
|
|
- type: nauc_map_at_1000_max
|
|
value: 27.445457824343595
|
|
- type: nauc_map_at_1000_std
|
|
value: -18.60670002314929
|
|
- type: nauc_map_at_100_diff1
|
|
value: 77.81776087022583
|
|
- type: nauc_map_at_100_max
|
|
value: 27.465677796741794
|
|
- type: nauc_map_at_100_std
|
|
value: -18.574455053179566
|
|
- type: nauc_map_at_10_diff1
|
|
value: 77.668921503636
|
|
- type: nauc_map_at_10_max
|
|
value: 27.564476726876563
|
|
- type: nauc_map_at_10_std
|
|
value: -18.67577233314456
|
|
- type: nauc_map_at_1_diff1
|
|
value: 80.13251752826227
|
|
- type: nauc_map_at_1_max
|
|
value: 19.700940114548352
|
|
- type: nauc_map_at_1_std
|
|
value: -24.276498497801104
|
|
- type: nauc_map_at_20_diff1
|
|
value: 77.76444686257037
|
|
- type: nauc_map_at_20_max
|
|
value: 27.507355610895434
|
|
- type: nauc_map_at_20_std
|
|
value: -18.570029885207234
|
|
- type: nauc_map_at_3_diff1
|
|
value: 77.62870706241021
|
|
- type: nauc_map_at_3_max
|
|
value: 25.979199504514654
|
|
- type: nauc_map_at_3_std
|
|
value: -20.480776195240768
|
|
- type: nauc_map_at_5_diff1
|
|
value: 77.68046637184071
|
|
- type: nauc_map_at_5_max
|
|
value: 27.068345296401887
|
|
- type: nauc_map_at_5_std
|
|
value: -19.515458511154968
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 78.12673001253819
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 28.23584877768183
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -17.765605843184606
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 78.12476632443614
|
|
- type: nauc_mrr_at_100_max
|
|
value: 28.255499574563654
|
|
- type: nauc_mrr_at_100_std
|
|
value: -17.73302695902061
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 77.98552897771079
|
|
- type: nauc_mrr_at_10_max
|
|
value: 28.433270245298903
|
|
- type: nauc_mrr_at_10_std
|
|
value: -17.721467674164725
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 80.74164178463916
|
|
- type: nauc_mrr_at_1_max
|
|
value: 23.400992011183135
|
|
- type: nauc_mrr_at_1_std
|
|
value: -23.155846305708668
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 78.08519488707572
|
|
- type: nauc_mrr_at_20_max
|
|
value: 28.305974768972476
|
|
- type: nauc_mrr_at_20_std
|
|
value: -17.70766096956611
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 77.99203426607973
|
|
- type: nauc_mrr_at_3_max
|
|
value: 27.39053740753677
|
|
- type: nauc_mrr_at_3_std
|
|
value: -19.110899565832597
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 77.99012861357085
|
|
- type: nauc_mrr_at_5_max
|
|
value: 28.018453732422905
|
|
- type: nauc_mrr_at_5_std
|
|
value: -18.45275089190139
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 77.37899152370498
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 29.715512454119402
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -15.311768186844196
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 77.30487512550962
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 30.358291073116767
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -14.276238712942787
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 76.55306779956729
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 31.003218536597576
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -14.528637377688142
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 80.74164178463916
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 23.400992011183135
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -23.155846305708668
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 76.92359358217516
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 30.734983558658648
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -14.12117266760052
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 76.65174056138369
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 27.744998584618365
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -18.596857381234265
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 76.64434516875298
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 29.580949778455096
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -16.820146947848347
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -15.819998326963425
|
|
- type: nauc_precision_at_1000_max
|
|
value: 22.790060032171432
|
|
- type: nauc_precision_at_1000_std
|
|
value: 25.646210332652032
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -3.225658983047692
|
|
- type: nauc_precision_at_100_max
|
|
value: 31.046785086458396
|
|
- type: nauc_precision_at_100_std
|
|
value: 30.64496213174489
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 22.399826113454544
|
|
- type: nauc_precision_at_10_max
|
|
value: 37.17215584865757
|
|
- type: nauc_precision_at_10_std
|
|
value: 16.375879066453813
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 80.74164178463916
|
|
- type: nauc_precision_at_1_max
|
|
value: 23.400992011183135
|
|
- type: nauc_precision_at_1_std
|
|
value: -23.155846305708668
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 11.824890141102545
|
|
- type: nauc_precision_at_20_max
|
|
value: 35.7858012680296
|
|
- type: nauc_precision_at_20_std
|
|
value: 24.36537306318588
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 46.964579254137156
|
|
- type: nauc_precision_at_3_max
|
|
value: 31.240508812172248
|
|
- type: nauc_precision_at_3_std
|
|
value: -4.790609954536406
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 35.92331054363029
|
|
- type: nauc_precision_at_5_max
|
|
value: 34.58921599366064
|
|
- type: nauc_precision_at_5_std
|
|
value: 3.955705927038542
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 69.82124326053469
|
|
- type: nauc_recall_at_1000_max
|
|
value: 77.26332872982017
|
|
- type: nauc_recall_at_1000_std
|
|
value: 74.20589405678723
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 71.09335151657598
|
|
- type: nauc_recall_at_100_max
|
|
value: 74.66551138520433
|
|
- type: nauc_recall_at_100_std
|
|
value: 62.296014312578606
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 68.34266216578438
|
|
- type: nauc_recall_at_10_max
|
|
value: 51.776074855673635
|
|
- type: nauc_recall_at_10_std
|
|
value: 11.551590635685633
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 80.13251752826227
|
|
- type: nauc_recall_at_1_max
|
|
value: 19.700940114548352
|
|
- type: nauc_recall_at_1_std
|
|
value: -24.276498497801104
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 68.44098404116468
|
|
- type: nauc_recall_at_20_max
|
|
value: 58.0709257934264
|
|
- type: nauc_recall_at_20_std
|
|
value: 27.20288447881239
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 72.224364274587
|
|
- type: nauc_recall_at_3_max
|
|
value: 32.11973511168104
|
|
- type: nauc_recall_at_3_std
|
|
value: -13.287781131985849
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 70.97684486885963
|
|
- type: nauc_recall_at_5_max
|
|
value: 39.47238239221433
|
|
- type: nauc_recall_at_5_std
|
|
value: -5.749985209368605
|
|
- type: ndcg_at_1
|
|
value: 67.55
|
|
- type: ndcg_at_10
|
|
value: 78.679
|
|
- type: ndcg_at_100
|
|
value: 80.16
|
|
- type: ndcg_at_1000
|
|
value: 80.42
|
|
- type: ndcg_at_20
|
|
value: 79.50500000000001
|
|
- type: ndcg_at_3
|
|
value: 74.96199999999999
|
|
- type: ndcg_at_5
|
|
value: 77.093
|
|
- type: precision_at_1
|
|
value: 67.55
|
|
- type: precision_at_10
|
|
value: 9.589
|
|
- type: precision_at_100
|
|
value: 1.031
|
|
- type: precision_at_1000
|
|
value: 0.105
|
|
- type: precision_at_20
|
|
value: 4.966
|
|
- type: precision_at_3
|
|
value: 28.319
|
|
- type: precision_at_5
|
|
value: 18.129
|
|
- type: recall_at_1
|
|
value: 65.349
|
|
- type: recall_at_10
|
|
value: 90.10000000000001
|
|
- type: recall_at_100
|
|
value: 96.685
|
|
- type: recall_at_1000
|
|
value: 98.714
|
|
- type: recall_at_20
|
|
value: 93.298
|
|
- type: recall_at_3
|
|
value: 80.324
|
|
- type: recall_at_5
|
|
value: 85.37700000000001
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: zh-CN
|
|
name: MTEB MassiveIntentClassification (zh-CN)
|
|
revision: 4672e20407010da34463acc759c162ca9734bca6
|
|
split: test
|
|
type: mteb/amazon_massive_intent
|
|
metrics:
|
|
- type: accuracy
|
|
value: 70.94149293880295
|
|
- type: f1
|
|
value: 67.43015916458866
|
|
- type: f1_weighted
|
|
value: 70.02165762549619
|
|
- type: main_score
|
|
value: 70.94149293880295
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: zh-TW
|
|
name: MTEB MassiveIntentClassification (zh-TW)
|
|
revision: 4672e20407010da34463acc759c162ca9734bca6
|
|
split: test
|
|
type: mteb/amazon_massive_intent
|
|
metrics:
|
|
- type: accuracy
|
|
value: 63.05312710154675
|
|
- type: f1
|
|
value: 61.11778922874984
|
|
- type: f1_weighted
|
|
value: 61.425454449692396
|
|
- type: main_score
|
|
value: 63.05312710154675
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: zh-CN
|
|
name: MTEB MassiveScenarioClassification (zh-CN)
|
|
revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
|
|
split: test
|
|
type: mteb/amazon_massive_scenario
|
|
metrics:
|
|
- type: accuracy
|
|
value: 77.79757901815736
|
|
- type: f1
|
|
value: 76.85610655879204
|
|
- type: f1_weighted
|
|
value: 77.36623686607157
|
|
- type: main_score
|
|
value: 77.79757901815736
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: zh-TW
|
|
name: MTEB MassiveScenarioClassification (zh-TW)
|
|
revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
|
|
split: test
|
|
type: mteb/amazon_massive_scenario
|
|
metrics:
|
|
- type: accuracy
|
|
value: 71.34498991257566
|
|
- type: f1
|
|
value: 71.42538497861686
|
|
- type: f1_weighted
|
|
value: 70.47776598531958
|
|
- type: main_score
|
|
value: 71.34498991257566
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB MedicalRetrieval
|
|
revision: 2039188fb5800a9803ba5048df7b76e6fb151fc6
|
|
split: dev
|
|
type: C-MTEB/MedicalRetrieval
|
|
metrics:
|
|
- type: main_score
|
|
value: 57.528999999999996
|
|
- type: map_at_1
|
|
value: 48.699999999999996
|
|
- type: map_at_10
|
|
value: 54.674
|
|
- type: map_at_100
|
|
value: 55.187
|
|
- type: map_at_1000
|
|
value: 55.24
|
|
- type: map_at_20
|
|
value: 54.933
|
|
- type: map_at_3
|
|
value: 53.367
|
|
- type: map_at_5
|
|
value: 54.081999999999994
|
|
- type: mrr_at_1
|
|
value: 48.8
|
|
- type: mrr_at_10
|
|
value: 54.71369047619046
|
|
- type: mrr_at_100
|
|
value: 55.23606881716415
|
|
- type: mrr_at_1000
|
|
value: 55.2887596380029
|
|
- type: mrr_at_20
|
|
value: 54.98226974307081
|
|
- type: mrr_at_3
|
|
value: 53.41666666666666
|
|
- type: mrr_at_5
|
|
value: 54.131666666666646
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 79.392997677128
|
|
- type: nauc_map_at_1000_max
|
|
value: 47.4042544614244
|
|
- type: nauc_map_at_1000_std
|
|
value: 23.2164546714886
|
|
- type: nauc_map_at_100_diff1
|
|
value: 79.3811285055918
|
|
- type: nauc_map_at_100_max
|
|
value: 47.399489637525214
|
|
- type: nauc_map_at_100_std
|
|
value: 23.24298678047571
|
|
- type: nauc_map_at_10_diff1
|
|
value: 79.51795702164893
|
|
- type: nauc_map_at_10_max
|
|
value: 47.3775323018549
|
|
- type: nauc_map_at_10_std
|
|
value: 22.863584607876017
|
|
- type: nauc_map_at_1_diff1
|
|
value: 82.77387889149895
|
|
- type: nauc_map_at_1_max
|
|
value: 48.92316018033766
|
|
- type: nauc_map_at_1_std
|
|
value: 20.670920881420933
|
|
- type: nauc_map_at_20_diff1
|
|
value: 79.36321354500926
|
|
- type: nauc_map_at_20_max
|
|
value: 47.347135287818695
|
|
- type: nauc_map_at_20_std
|
|
value: 23.128792587733724
|
|
- type: nauc_map_at_3_diff1
|
|
value: 79.89693675044646
|
|
- type: nauc_map_at_3_max
|
|
value: 47.999519454025815
|
|
- type: nauc_map_at_3_std
|
|
value: 22.67285215587248
|
|
- type: nauc_map_at_5_diff1
|
|
value: 79.72880868956226
|
|
- type: nauc_map_at_5_max
|
|
value: 47.870829359727615
|
|
- type: nauc_map_at_5_std
|
|
value: 22.75976001331719
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 79.2558524289943
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 47.68193948210489
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 23.488171939833503
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 79.2441760972466
|
|
- type: nauc_mrr_at_100_max
|
|
value: 47.67677923765432
|
|
- type: nauc_mrr_at_100_std
|
|
value: 23.51432250784555
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 79.39423493974832
|
|
- type: nauc_mrr_at_10_max
|
|
value: 47.672297066929545
|
|
- type: nauc_mrr_at_10_std
|
|
value: 23.13845505800058
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 82.51854957699533
|
|
- type: nauc_mrr_at_1_max
|
|
value: 49.43475537911197
|
|
- type: nauc_mrr_at_1_std
|
|
value: 21.172657021240443
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 79.22702612117199
|
|
- type: nauc_mrr_at_20_max
|
|
value: 47.62286080846738
|
|
- type: nauc_mrr_at_20_std
|
|
value: 23.398587017649174
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 79.76301529177348
|
|
- type: nauc_mrr_at_3_max
|
|
value: 48.26663425470944
|
|
- type: nauc_mrr_at_3_std
|
|
value: 22.935349467987145
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 79.5934610019844
|
|
- type: nauc_mrr_at_5_max
|
|
value: 48.1407033814883
|
|
- type: nauc_mrr_at_5_std
|
|
value: 23.025008156084695
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 77.97548063568358
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 46.670156188276266
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 25.32524568996684
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 77.58788261282791
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 46.366231150510664
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 26.02842093987038
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 78.15883898742274
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 46.181496192291974
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 23.997358704992077
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 82.77387889149895
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 48.92316018033766
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 20.670920881420933
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 77.51209948232727
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 46.02903895633775
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 25.023178998194467
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 79.0464751622174
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 47.65456262552185
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 23.50005981191216
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 78.73621060890696
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 47.4490746627881
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 23.70727530773819
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 63.42066238259988
|
|
- type: nauc_precision_at_1000_max
|
|
value: 43.54369198659821
|
|
- type: nauc_precision_at_1000_std
|
|
value: 55.676388202339524
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 67.14856074074835
|
|
- type: nauc_precision_at_100_max
|
|
value: 40.92023184354666
|
|
- type: nauc_precision_at_100_std
|
|
value: 45.790641988757145
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 73.22243545156664
|
|
- type: nauc_precision_at_10_max
|
|
value: 41.458823923773686
|
|
- type: nauc_precision_at_10_std
|
|
value: 28.142697919198138
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 82.77387889149895
|
|
- type: nauc_precision_at_1_max
|
|
value: 48.92316018033766
|
|
- type: nauc_precision_at_1_std
|
|
value: 20.670920881420933
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 69.5822714276579
|
|
- type: nauc_precision_at_20_max
|
|
value: 40.258145844180724
|
|
- type: nauc_precision_at_20_std
|
|
value: 33.443132096498665
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 76.48729951428531
|
|
- type: nauc_precision_at_3_max
|
|
value: 46.58972515297812
|
|
- type: nauc_precision_at_3_std
|
|
value: 26.07700999310317
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 75.58859746051998
|
|
- type: nauc_precision_at_5_max
|
|
value: 46.09484444567729
|
|
- type: nauc_precision_at_5_std
|
|
value: 26.82420134602608
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 63.42066238260002
|
|
- type: nauc_recall_at_1000_max
|
|
value: 43.543691986598645
|
|
- type: nauc_recall_at_1000_std
|
|
value: 55.67638820233998
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 67.14856074074834
|
|
- type: nauc_recall_at_100_max
|
|
value: 40.92023184354673
|
|
- type: nauc_recall_at_100_std
|
|
value: 45.79064198875728
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 73.22243545156665
|
|
- type: nauc_recall_at_10_max
|
|
value: 41.45882392377375
|
|
- type: nauc_recall_at_10_std
|
|
value: 28.14269791919819
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 82.77387889149895
|
|
- type: nauc_recall_at_1_max
|
|
value: 48.92316018033766
|
|
- type: nauc_recall_at_1_std
|
|
value: 20.670920881420933
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 69.58227142765797
|
|
- type: nauc_recall_at_20_max
|
|
value: 40.25814584418081
|
|
- type: nauc_recall_at_20_std
|
|
value: 33.443132096498665
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 76.4872995142853
|
|
- type: nauc_recall_at_3_max
|
|
value: 46.589725152978076
|
|
- type: nauc_recall_at_3_std
|
|
value: 26.07700999310312
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 75.58859746051999
|
|
- type: nauc_recall_at_5_max
|
|
value: 46.09484444567737
|
|
- type: nauc_recall_at_5_std
|
|
value: 26.8242013460261
|
|
- type: ndcg_at_1
|
|
value: 48.699999999999996
|
|
- type: ndcg_at_10
|
|
value: 57.528999999999996
|
|
- type: ndcg_at_100
|
|
value: 60.38
|
|
- type: ndcg_at_1000
|
|
value: 61.937
|
|
- type: ndcg_at_20
|
|
value: 58.518
|
|
- type: ndcg_at_3
|
|
value: 54.818999999999996
|
|
- type: ndcg_at_5
|
|
value: 56.101
|
|
- type: precision_at_1
|
|
value: 48.699999999999996
|
|
- type: precision_at_10
|
|
value: 6.65
|
|
- type: precision_at_100
|
|
value: 0.8059999999999999
|
|
- type: precision_at_1000
|
|
value: 0.093
|
|
- type: precision_at_20
|
|
value: 3.5249999999999995
|
|
- type: precision_at_3
|
|
value: 19.667
|
|
- type: precision_at_5
|
|
value: 12.42
|
|
- type: recall_at_1
|
|
value: 48.699999999999996
|
|
- type: recall_at_10
|
|
value: 66.5
|
|
- type: recall_at_100
|
|
value: 80.60000000000001
|
|
- type: recall_at_1000
|
|
value: 93.2
|
|
- type: recall_at_20
|
|
value: 70.5
|
|
- type: recall_at_3
|
|
value: 59.0
|
|
- type: recall_at_5
|
|
value: 62.1
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB MultilingualSentiment
|
|
revision: 46958b007a63fdbf239b7672c25d0bea67b5ea1a
|
|
split: test
|
|
type: C-MTEB/MultilingualSentiment-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 78.08
|
|
- type: f1
|
|
value: 77.44308848942492
|
|
- type: f1_weighted
|
|
value: 77.44308848942492
|
|
- type: main_score
|
|
value: 78.08
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB Ocnli
|
|
revision: 66e76a618a34d6d565d5538088562851e6daa7ec
|
|
split: validation
|
|
type: C-MTEB/OCNLI
|
|
metrics:
|
|
- type: cosine_accuracy
|
|
value: 66.8651867893882
|
|
- type: cosine_accuracy_threshold
|
|
value: 84.34688448905945
|
|
- type: cosine_ap
|
|
value: 69.83287846115917
|
|
- type: cosine_f1
|
|
value: 71.33520074696546
|
|
- type: cosine_f1_threshold
|
|
value: 83.85992050170898
|
|
- type: cosine_precision
|
|
value: 63.93305439330545
|
|
- type: cosine_recall
|
|
value: 80.67581837381204
|
|
- type: dot_accuracy
|
|
value: 66.8651867893882
|
|
- type: dot_accuracy_threshold
|
|
value: 84.34690237045288
|
|
- type: dot_ap
|
|
value: 69.83287846115917
|
|
- type: dot_f1
|
|
value: 71.33520074696546
|
|
- type: dot_f1_threshold
|
|
value: 83.85992050170898
|
|
- type: dot_precision
|
|
value: 63.93305439330545
|
|
- type: dot_recall
|
|
value: 80.67581837381204
|
|
- type: euclidean_accuracy
|
|
value: 66.8651867893882
|
|
- type: euclidean_accuracy_threshold
|
|
value: 55.95196485519409
|
|
- type: euclidean_ap
|
|
value: 69.83287846115917
|
|
- type: euclidean_f1
|
|
value: 71.33520074696546
|
|
- type: euclidean_f1_threshold
|
|
value: 56.81561827659607
|
|
- type: euclidean_precision
|
|
value: 63.93305439330545
|
|
- type: euclidean_recall
|
|
value: 80.67581837381204
|
|
- type: main_score
|
|
value: 69.83287846115917
|
|
- type: manhattan_accuracy
|
|
value: 66.0530590146183
|
|
- type: manhattan_accuracy_threshold
|
|
value: 1215.458583831787
|
|
- type: manhattan_ap
|
|
value: 69.51465499538298
|
|
- type: manhattan_f1
|
|
value: 70.56159420289853
|
|
- type: manhattan_f1_threshold
|
|
value: 1344.7942733764648
|
|
- type: manhattan_precision
|
|
value: 61.77636796193497
|
|
- type: manhattan_recall
|
|
value: 82.259767687434
|
|
- type: max_ap
|
|
value: 69.83287846115917
|
|
- type: max_f1
|
|
value: 71.33520074696546
|
|
- type: max_precision
|
|
value: 63.93305439330545
|
|
- type: max_recall
|
|
value: 82.259767687434
|
|
- type: similarity_accuracy
|
|
value: 66.8651867893882
|
|
- type: similarity_accuracy_threshold
|
|
value: 84.34688448905945
|
|
- type: similarity_ap
|
|
value: 69.83287846115917
|
|
- type: similarity_f1
|
|
value: 71.33520074696546
|
|
- type: similarity_f1_threshold
|
|
value: 83.85992050170898
|
|
- type: similarity_precision
|
|
value: 63.93305439330545
|
|
- type: similarity_recall
|
|
value: 80.67581837381204
|
|
task:
|
|
type: PairClassification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB OnlineShopping
|
|
revision: e610f2ebd179a8fda30ae534c3878750a96db120
|
|
split: test
|
|
type: C-MTEB/OnlineShopping-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 93.66999999999999
|
|
- type: ap
|
|
value: 92.68160375501351
|
|
- type: ap_weighted
|
|
value: 92.68160375501351
|
|
- type: f1
|
|
value: 93.6673524115384
|
|
- type: f1_weighted
|
|
value: 93.67269842799493
|
|
- type: main_score
|
|
value: 93.66999999999999
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB PAWSX
|
|
revision: 9c6a90e430ac22b5779fb019a23e820b11a8b5e1
|
|
split: test
|
|
type: C-MTEB/PAWSX
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 14.427978400689973
|
|
- type: cosine_spearman
|
|
value: 15.182736434509348
|
|
- type: euclidean_pearson
|
|
value: 17.726048874983753
|
|
- type: euclidean_spearman
|
|
value: 15.201779286945575
|
|
- type: main_score
|
|
value: 15.182736434509348
|
|
- type: manhattan_pearson
|
|
value: 17.715716154164234
|
|
- type: manhattan_spearman
|
|
value: 15.250986981738777
|
|
- type: pearson
|
|
value: 14.427978400689973
|
|
- type: spearman
|
|
value: 15.182736434509348
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB QBQTC
|
|
revision: 790b0510dc52b1553e8c49f3d2afb48c0e5c48b7
|
|
split: test
|
|
type: C-MTEB/QBQTC
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 28.677852039385687
|
|
- type: cosine_spearman
|
|
value: 30.317414500566187
|
|
- type: euclidean_pearson
|
|
value: 28.546943523039168
|
|
- type: euclidean_spearman
|
|
value: 30.31773442605619
|
|
- type: main_score
|
|
value: 30.317414500566187
|
|
- type: manhattan_pearson
|
|
value: 29.06524931618951
|
|
- type: manhattan_spearman
|
|
value: 30.85475318983088
|
|
- type: pearson
|
|
value: 28.677852039385687
|
|
- type: spearman
|
|
value: 30.317414500566187
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: zh
|
|
name: MTEB STS22 (zh)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 74.25169654144152
|
|
- type: cosine_spearman
|
|
value: 74.02188505990078
|
|
- type: euclidean_pearson
|
|
value: 71.78459076777199
|
|
- type: euclidean_spearman
|
|
value: 74.02188505990078
|
|
- type: main_score
|
|
value: 74.02188505990078
|
|
- type: manhattan_pearson
|
|
value: 71.38471936226554
|
|
- type: manhattan_spearman
|
|
value: 73.72453020549669
|
|
- type: pearson
|
|
value: 74.25169654144152
|
|
- type: spearman
|
|
value: 74.02188505990078
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: zh-en
|
|
name: MTEB STS22 (zh-en)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 77.21035942564082
|
|
- type: cosine_spearman
|
|
value: 76.57212143103963
|
|
- type: euclidean_pearson
|
|
value: 78.03973868360728
|
|
- type: euclidean_spearman
|
|
value: 76.57212143103963
|
|
- type: main_score
|
|
value: 76.57212143103963
|
|
- type: manhattan_pearson
|
|
value: 78.16591898142042
|
|
- type: manhattan_spearman
|
|
value: 76.83958214147293
|
|
- type: pearson
|
|
value: 77.21035942564082
|
|
- type: spearman
|
|
value: 76.57212143103963
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB STSB
|
|
revision: 0cde68302b3541bb8b3c340dc0644b0b745b3dc0
|
|
split: test
|
|
type: C-MTEB/STSB
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 76.73366278962006
|
|
- type: cosine_spearman
|
|
value: 78.136597096582
|
|
- type: euclidean_pearson
|
|
value: 77.15227584574502
|
|
- type: euclidean_spearman
|
|
value: 78.13622498113003
|
|
- type: main_score
|
|
value: 78.136597096582
|
|
- type: manhattan_pearson
|
|
value: 77.02225035694117
|
|
- type: manhattan_spearman
|
|
value: 78.03964720563964
|
|
- type: pearson
|
|
value: 76.73366278962006
|
|
- type: spearman
|
|
value: 78.136597096582
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB T2Reranking
|
|
revision: 76631901a18387f85eaa53e5450019b87ad58ef9
|
|
split: dev
|
|
type: C-MTEB/T2Reranking
|
|
metrics:
|
|
- type: main_score
|
|
value: 66.38154648171584
|
|
- type: map
|
|
value: 66.38154648171584
|
|
- type: mrr
|
|
value: 76.14530606871499
|
|
- type: nAUC_map_diff1
|
|
value: -9.806394737932642
|
|
- type: nAUC_map_max
|
|
value: 33.96115791248053
|
|
- type: nAUC_map_std
|
|
value: -3.643316859964786
|
|
- type: nAUC_mrr_diff1
|
|
value: -6.510263484170889
|
|
- type: nAUC_mrr_max
|
|
value: 26.441557887574124
|
|
- type: nAUC_mrr_std
|
|
value: -4.608018494327204
|
|
task:
|
|
type: Reranking
|
|
- dataset:
|
|
config: default
|
|
name: MTEB T2Retrieval
|
|
revision: 8731a845f1bf500a4f111cf1070785c793d10e64
|
|
split: dev
|
|
type: C-MTEB/T2Retrieval
|
|
metrics:
|
|
- type: main_score
|
|
value: 84.133
|
|
- type: map_at_1
|
|
value: 27.297
|
|
- type: map_at_10
|
|
value: 76.494
|
|
- type: map_at_100
|
|
value: 80.119
|
|
- type: map_at_1000
|
|
value: 80.185
|
|
- type: map_at_20
|
|
value: 79.251
|
|
- type: map_at_3
|
|
value: 53.864999999999995
|
|
- type: map_at_5
|
|
value: 66.143
|
|
- type: mrr_at_1
|
|
value: 89.57566193231632
|
|
- type: mrr_at_10
|
|
value: 92.13000711126722
|
|
- type: mrr_at_100
|
|
value: 92.21882184581148
|
|
- type: mrr_at_1000
|
|
value: 92.22214774256558
|
|
- type: mrr_at_20
|
|
value: 92.18699134744894
|
|
- type: mrr_at_3
|
|
value: 91.66228300894257
|
|
- type: mrr_at_5
|
|
value: 91.97264597580231
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 15.207460819974095
|
|
- type: nauc_map_at_1000_max
|
|
value: 42.32453165892631
|
|
- type: nauc_map_at_1000_std
|
|
value: 21.593634336302127
|
|
- type: nauc_map_at_100_diff1
|
|
value: 15.216272171820561
|
|
- type: nauc_map_at_100_max
|
|
value: 42.22983840076597
|
|
- type: nauc_map_at_100_std
|
|
value: 21.534370324932652
|
|
- type: nauc_map_at_10_diff1
|
|
value: 19.599553856210008
|
|
- type: nauc_map_at_10_max
|
|
value: 30.246318219245573
|
|
- type: nauc_map_at_10_std
|
|
value: 5.914404965156733
|
|
- type: nauc_map_at_1_diff1
|
|
value: 52.87085305237716
|
|
- type: nauc_map_at_1_max
|
|
value: -24.27989564325726
|
|
- type: nauc_map_at_1_std
|
|
value: -35.442050298290376
|
|
- type: nauc_map_at_20_diff1
|
|
value: 15.87998380728732
|
|
- type: nauc_map_at_20_max
|
|
value: 39.78308211411551
|
|
- type: nauc_map_at_20_std
|
|
value: 18.241218939315434
|
|
- type: nauc_map_at_3_diff1
|
|
value: 39.155089053329014
|
|
- type: nauc_map_at_3_max
|
|
value: -11.970155586820502
|
|
- type: nauc_map_at_3_std
|
|
value: -31.83333979404834
|
|
- type: nauc_map_at_5_diff1
|
|
value: 31.43539185744996
|
|
- type: nauc_map_at_5_max
|
|
value: 3.5586067754503152
|
|
- type: nauc_map_at_5_std
|
|
value: -20.89939723260621
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 47.58856242843391
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 73.33044542878086
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 41.41370720044016
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 47.58885589082642
|
|
- type: nauc_mrr_at_100_max
|
|
value: 73.33895048178488
|
|
- type: nauc_mrr_at_100_std
|
|
value: 41.42862248729776
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 47.60432720674615
|
|
- type: nauc_mrr_at_10_max
|
|
value: 73.47964069672504
|
|
- type: nauc_mrr_at_10_std
|
|
value: 41.60604407817306
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 47.84195771830615
|
|
- type: nauc_mrr_at_1_max
|
|
value: 68.95221045759685
|
|
- type: nauc_mrr_at_1_std
|
|
value: 35.145250281429824
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 47.58534671931297
|
|
- type: nauc_mrr_at_20_max
|
|
value: 73.39618815713096
|
|
- type: nauc_mrr_at_20_std
|
|
value: 41.50538366605475
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 47.54080143480509
|
|
- type: nauc_mrr_at_3_max
|
|
value: 73.27456449852177
|
|
- type: nauc_mrr_at_3_std
|
|
value: 41.190010138623364
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 47.631799071300314
|
|
- type: nauc_mrr_at_5_max
|
|
value: 73.50427384392508
|
|
- type: nauc_mrr_at_5_std
|
|
value: 41.41445819292792
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 19.178203338132032
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 54.846002008332206
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 33.669755579706234
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 18.825625578528154
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 53.96154830438667
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 33.63879617215427
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 18.95559446945268
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 44.21334528575739
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 22.47737214494352
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 47.84195771830615
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 68.95221045759685
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 35.145250281429824
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 18.915787332802143
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 48.64628634208606
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 27.471901227649102
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 14.800326460175548
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 58.714123081214986
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 32.87146819333138
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 15.117887863548916
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 51.62270126506565
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 28.21637936542305
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -34.6115257538737
|
|
- type: nauc_precision_at_1000_max
|
|
value: 46.57505454335497
|
|
- type: nauc_precision_at_1000_std
|
|
value: 58.73410354296305
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -34.51864090348213
|
|
- type: nauc_precision_at_100_max
|
|
value: 48.12778307352527
|
|
- type: nauc_precision_at_100_std
|
|
value: 60.33112526548986
|
|
- type: nauc_precision_at_10_diff1
|
|
value: -33.913446995683536
|
|
- type: nauc_precision_at_10_max
|
|
value: 51.827800576762726
|
|
- type: nauc_precision_at_10_std
|
|
value: 56.15214316846719
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 47.84195771830615
|
|
- type: nauc_precision_at_1_max
|
|
value: 68.95221045759685
|
|
- type: nauc_precision_at_1_std
|
|
value: 35.145250281429824
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -34.25535498799855
|
|
- type: nauc_precision_at_20_max
|
|
value: 50.23119733433027
|
|
- type: nauc_precision_at_20_std
|
|
value: 59.671418737988546
|
|
- type: nauc_precision_at_3_diff1
|
|
value: -28.417107232598877
|
|
- type: nauc_precision_at_3_max
|
|
value: 61.16886341335774
|
|
- type: nauc_precision_at_3_std
|
|
value: 48.34533128391697
|
|
- type: nauc_precision_at_5_diff1
|
|
value: -33.54570066440394
|
|
- type: nauc_precision_at_5_max
|
|
value: 56.522769824532936
|
|
- type: nauc_precision_at_5_std
|
|
value: 51.704950593707935
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 2.93977183499487
|
|
- type: nauc_recall_at_1000_max
|
|
value: 59.19161397622145
|
|
- type: nauc_recall_at_1000_std
|
|
value: 62.44563668374114
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 8.013825549311562
|
|
- type: nauc_recall_at_100_max
|
|
value: 49.846341160862714
|
|
- type: nauc_recall_at_100_std
|
|
value: 48.1170998033127
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 18.010735796887985
|
|
- type: nauc_recall_at_10_max
|
|
value: 21.358569425898903
|
|
- type: nauc_recall_at_10_std
|
|
value: 1.3301139186106035
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 52.87085305237716
|
|
- type: nauc_recall_at_1_max
|
|
value: -24.27989564325726
|
|
- type: nauc_recall_at_1_std
|
|
value: -35.442050298290376
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 11.816321531579238
|
|
- type: nauc_recall_at_20_max
|
|
value: 36.13782953010234
|
|
- type: nauc_recall_at_20_std
|
|
value: 23.555109581359886
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 37.46336191367832
|
|
- type: nauc_recall_at_3_max
|
|
value: -16.038670342884316
|
|
- type: nauc_recall_at_3_std
|
|
value: -34.074784083025214
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 30.274716744272567
|
|
- type: nauc_recall_at_5_max
|
|
value: -4.34067124108913
|
|
- type: nauc_recall_at_5_std
|
|
value: -26.21894992157237
|
|
- type: ndcg_at_1
|
|
value: 89.576
|
|
- type: ndcg_at_10
|
|
value: 84.133
|
|
- type: ndcg_at_100
|
|
value: 87.773
|
|
- type: ndcg_at_1000
|
|
value: 88.421
|
|
- type: ndcg_at_20
|
|
value: 85.909
|
|
- type: ndcg_at_3
|
|
value: 85.539
|
|
- type: ndcg_at_5
|
|
value: 84.143
|
|
- type: precision_at_1
|
|
value: 89.576
|
|
- type: precision_at_10
|
|
value: 41.789
|
|
- type: precision_at_100
|
|
value: 4.995
|
|
- type: precision_at_1000
|
|
value: 0.515
|
|
- type: precision_at_20
|
|
value: 23.224
|
|
- type: precision_at_3
|
|
value: 74.79400000000001
|
|
- type: precision_at_5
|
|
value: 62.683
|
|
- type: recall_at_1
|
|
value: 27.297
|
|
- type: recall_at_10
|
|
value: 83.035
|
|
- type: recall_at_100
|
|
value: 94.915
|
|
- type: recall_at_1000
|
|
value: 98.225
|
|
- type: recall_at_20
|
|
value: 88.984
|
|
- type: recall_at_3
|
|
value: 55.533
|
|
- type: recall_at_5
|
|
value: 69.575
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB TNews
|
|
revision: 317f262bf1e6126357bbe89e875451e4b0938fe4
|
|
split: validation
|
|
type: C-MTEB/TNews-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 51.664
|
|
- type: f1
|
|
value: 49.254634831292336
|
|
- type: f1_weighted
|
|
value: 51.23047453836118
|
|
- type: main_score
|
|
value: 51.664
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB ThuNewsClusteringP2P
|
|
revision: 5798586b105c0434e4f0fe5e767abe619442cf93
|
|
split: test
|
|
type: C-MTEB/ThuNewsClusteringP2P
|
|
metrics:
|
|
- type: main_score
|
|
value: 62.931149356482294
|
|
- type: v_measure
|
|
value: 62.931149356482294
|
|
- type: v_measure_std
|
|
value: 1.2113879267357022
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB ThuNewsClusteringS2S
|
|
revision: 8a8b2caeda43f39e13c4bc5bea0f8a667896e10d
|
|
split: test
|
|
type: C-MTEB/ThuNewsClusteringS2S
|
|
metrics:
|
|
- type: main_score
|
|
value: 59.18138500076393
|
|
- type: v_measure
|
|
value: 59.18138500076393
|
|
- type: v_measure_std
|
|
value: 1.441163494106974
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB VideoRetrieval
|
|
revision: 58c2597a5943a2ba48f4668c3b90d796283c5639
|
|
split: dev
|
|
type: C-MTEB/VideoRetrieval
|
|
metrics:
|
|
- type: main_score
|
|
value: 72.14500000000001
|
|
- type: map_at_1
|
|
value: 58.8
|
|
- type: map_at_10
|
|
value: 68.014
|
|
- type: map_at_100
|
|
value: 68.506
|
|
- type: map_at_1000
|
|
value: 68.51899999999999
|
|
- type: map_at_20
|
|
value: 68.333
|
|
- type: map_at_3
|
|
value: 66.31700000000001
|
|
- type: map_at_5
|
|
value: 67.31200000000001
|
|
- type: mrr_at_1
|
|
value: 58.8
|
|
- type: mrr_at_10
|
|
value: 68.01432539682544
|
|
- type: mrr_at_100
|
|
value: 68.50595347947811
|
|
- type: mrr_at_1000
|
|
value: 68.51919475199976
|
|
- type: mrr_at_20
|
|
value: 68.33299226014789
|
|
- type: mrr_at_3
|
|
value: 66.3166666666667
|
|
- type: mrr_at_5
|
|
value: 67.31166666666671
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 68.3842603726721
|
|
- type: nauc_map_at_1000_max
|
|
value: 5.841784848188991
|
|
- type: nauc_map_at_1000_std
|
|
value: -31.890361063810364
|
|
- type: nauc_map_at_100_diff1
|
|
value: 68.38290538651279
|
|
- type: nauc_map_at_100_max
|
|
value: 5.851346250195991
|
|
- type: nauc_map_at_100_std
|
|
value: -31.88363804217233
|
|
- type: nauc_map_at_10_diff1
|
|
value: 68.42162270332948
|
|
- type: nauc_map_at_10_max
|
|
value: 5.545878771437991
|
|
- type: nauc_map_at_10_std
|
|
value: -32.33063386887081
|
|
- type: nauc_map_at_1_diff1
|
|
value: 69.28202470263717
|
|
- type: nauc_map_at_1_max
|
|
value: 5.263512365959786
|
|
- type: nauc_map_at_1_std
|
|
value: -29.659416343096055
|
|
- type: nauc_map_at_20_diff1
|
|
value: 68.3987969552634
|
|
- type: nauc_map_at_20_max
|
|
value: 5.7847092517499785
|
|
- type: nauc_map_at_20_std
|
|
value: -32.0616280955644
|
|
- type: nauc_map_at_3_diff1
|
|
value: 68.42478821018057
|
|
- type: nauc_map_at_3_max
|
|
value: 4.861120340503774
|
|
- type: nauc_map_at_3_std
|
|
value: -33.56938270962587
|
|
- type: nauc_map_at_5_diff1
|
|
value: 68.20507686427763
|
|
- type: nauc_map_at_5_max
|
|
value: 5.369798374942801
|
|
- type: nauc_map_at_5_std
|
|
value: -32.83081659270383
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 68.3842603726721
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 5.841784848188991
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -31.890361063810364
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 68.38290538651279
|
|
- type: nauc_mrr_at_100_max
|
|
value: 5.851346250195991
|
|
- type: nauc_mrr_at_100_std
|
|
value: -31.88363804217233
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 68.42162270332948
|
|
- type: nauc_mrr_at_10_max
|
|
value: 5.545878771437991
|
|
- type: nauc_mrr_at_10_std
|
|
value: -32.33063386887081
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 69.28202470263717
|
|
- type: nauc_mrr_at_1_max
|
|
value: 5.263512365959786
|
|
- type: nauc_mrr_at_1_std
|
|
value: -29.659416343096055
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 68.3987969552634
|
|
- type: nauc_mrr_at_20_max
|
|
value: 5.7847092517499785
|
|
- type: nauc_mrr_at_20_std
|
|
value: -32.0616280955644
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 68.42478821018057
|
|
- type: nauc_mrr_at_3_max
|
|
value: 4.861120340503774
|
|
- type: nauc_mrr_at_3_std
|
|
value: -33.56938270962587
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 68.20507686427763
|
|
- type: nauc_mrr_at_5_max
|
|
value: 5.369798374942801
|
|
- type: nauc_mrr_at_5_std
|
|
value: -32.83081659270383
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 68.14552912036231
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 7.562355001802865
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -30.13999419402607
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 68.09990028004812
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 7.917285926128676
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -29.909889861196902
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 68.32387598538823
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 6.442888130533218
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -32.43505234576926
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 69.28202470263717
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 5.263512365959786
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -29.659416343096055
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 68.19058463118989
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 7.4710128713487975
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -31.212367402512527
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 68.2422738747729
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 4.866392479207864
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -35.0611297009806
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 67.76867006392196
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 5.876702580928499
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -33.66450752679279
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 59.01318860877509
|
|
- type: nauc_precision_at_1000_max
|
|
value: 92.88340336134347
|
|
- type: nauc_precision_at_1000_std
|
|
value: 92.92425303454743
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 62.909039584826274
|
|
- type: nauc_precision_at_100_max
|
|
value: 53.748941437039655
|
|
- type: nauc_precision_at_100_std
|
|
value: 25.24916943521579
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 68.09729905629663
|
|
- type: nauc_precision_at_10_max
|
|
value: 12.03384315001613
|
|
- type: nauc_precision_at_10_std
|
|
value: -31.81483891962282
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 69.28202470263717
|
|
- type: nauc_precision_at_1_max
|
|
value: 5.263512365959786
|
|
- type: nauc_precision_at_1_std
|
|
value: -29.659416343096055
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 66.6897634037554
|
|
- type: nauc_precision_at_20_max
|
|
value: 23.11402140195658
|
|
- type: nauc_precision_at_20_std
|
|
value: -20.564049852242167
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 67.64170624528396
|
|
- type: nauc_precision_at_3_max
|
|
value: 4.945160628945999
|
|
- type: nauc_precision_at_3_std
|
|
value: -40.41499950328566
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 65.92840910208848
|
|
- type: nauc_precision_at_5_max
|
|
value: 8.229706730154186
|
|
- type: nauc_precision_at_5_std
|
|
value: -36.74013989591443
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 59.01318860877662
|
|
- type: nauc_recall_at_1000_max
|
|
value: 92.88340336134418
|
|
- type: nauc_recall_at_1000_std
|
|
value: 92.92425303454706
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 62.90903958482619
|
|
- type: nauc_recall_at_100_max
|
|
value: 53.748941437040145
|
|
- type: nauc_recall_at_100_std
|
|
value: 25.249169435216018
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 68.0972990562968
|
|
- type: nauc_recall_at_10_max
|
|
value: 12.033843150016319
|
|
- type: nauc_recall_at_10_std
|
|
value: -31.814838919622566
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 69.28202470263717
|
|
- type: nauc_recall_at_1_max
|
|
value: 5.263512365959786
|
|
- type: nauc_recall_at_1_std
|
|
value: -29.659416343096055
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 66.6897634037554
|
|
- type: nauc_recall_at_20_max
|
|
value: 23.114021401956656
|
|
- type: nauc_recall_at_20_std
|
|
value: -20.564049852241986
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 67.64170624528384
|
|
- type: nauc_recall_at_3_max
|
|
value: 4.9451606289460095
|
|
- type: nauc_recall_at_3_std
|
|
value: -40.41499950328563
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 65.92840910208865
|
|
- type: nauc_recall_at_5_max
|
|
value: 8.229706730154424
|
|
- type: nauc_recall_at_5_std
|
|
value: -36.740139895914325
|
|
- type: ndcg_at_1
|
|
value: 58.8
|
|
- type: ndcg_at_10
|
|
value: 72.14500000000001
|
|
- type: ndcg_at_100
|
|
value: 74.477
|
|
- type: ndcg_at_1000
|
|
value: 74.821
|
|
- type: ndcg_at_20
|
|
value: 73.34
|
|
- type: ndcg_at_3
|
|
value: 68.634
|
|
- type: ndcg_at_5
|
|
value: 70.416
|
|
- type: precision_at_1
|
|
value: 58.8
|
|
- type: precision_at_10
|
|
value: 8.5
|
|
- type: precision_at_100
|
|
value: 0.9570000000000001
|
|
- type: precision_at_1000
|
|
value: 0.098
|
|
- type: precision_at_20
|
|
value: 4.49
|
|
- type: precision_at_3
|
|
value: 25.1
|
|
- type: precision_at_5
|
|
value: 15.920000000000002
|
|
- type: recall_at_1
|
|
value: 58.8
|
|
- type: recall_at_10
|
|
value: 85.0
|
|
- type: recall_at_100
|
|
value: 95.7
|
|
- type: recall_at_1000
|
|
value: 98.4
|
|
- type: recall_at_20
|
|
value: 89.8
|
|
- type: recall_at_3
|
|
value: 75.3
|
|
- type: recall_at_5
|
|
value: 79.60000000000001
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB Waimai
|
|
revision: 339287def212450dcaa9df8c22bf93e9980c7023
|
|
split: test
|
|
type: C-MTEB/waimai-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 89.12
|
|
- type: ap
|
|
value: 74.85094489946682
|
|
- type: ap_weighted
|
|
value: 74.85094489946682
|
|
- type: f1
|
|
value: 87.58964139879481
|
|
- type: f1_weighted
|
|
value: 89.11267843686537
|
|
- type: main_score
|
|
value: 89.12
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB AlloProfClusteringP2P
|
|
revision: 392ba3f5bcc8c51f578786c1fc3dae648662cb9b
|
|
split: test
|
|
type: lyon-nlp/alloprof
|
|
metrics:
|
|
- type: main_score
|
|
value: 66.7100274116735
|
|
- type: v_measure
|
|
value: 66.7100274116735
|
|
- type: v_measure_std
|
|
value: 2.065600197695283
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB AlloProfClusteringS2S
|
|
revision: 392ba3f5bcc8c51f578786c1fc3dae648662cb9b
|
|
split: test
|
|
type: lyon-nlp/alloprof
|
|
metrics:
|
|
- type: main_score
|
|
value: 47.67572024379311
|
|
- type: v_measure
|
|
value: 47.67572024379311
|
|
- type: v_measure_std
|
|
value: 3.1905282169494953
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB AlloprofReranking
|
|
revision: 65393d0d7a08a10b4e348135e824f385d420b0fd
|
|
split: test
|
|
type: lyon-nlp/mteb-fr-reranking-alloprof-s2p
|
|
metrics:
|
|
- type: main_score
|
|
value: 75.04647907753767
|
|
- type: map
|
|
value: 75.04647907753767
|
|
- type: mrr
|
|
value: 76.25801875154207
|
|
- type: nAUC_map_diff1
|
|
value: 56.38279442235466
|
|
- type: nAUC_map_max
|
|
value: 20.009630947768642
|
|
- type: nAUC_map_std
|
|
value: 21.626818227466185
|
|
- type: nAUC_mrr_diff1
|
|
value: 56.33463291672874
|
|
- type: nAUC_mrr_max
|
|
value: 20.472794140230853
|
|
- type: nAUC_mrr_std
|
|
value: 21.491759650866392
|
|
task:
|
|
type: Reranking
|
|
- dataset:
|
|
config: default
|
|
name: MTEB AlloprofRetrieval
|
|
revision: fcf295ea64c750f41fadbaa37b9b861558e1bfbd
|
|
split: test
|
|
type: lyon-nlp/alloprof
|
|
metrics:
|
|
- type: main_score
|
|
value: 50.638000000000005
|
|
- type: map_at_1
|
|
value: 33.161
|
|
- type: map_at_10
|
|
value: 44.698
|
|
- type: map_at_100
|
|
value: 45.596
|
|
- type: map_at_1000
|
|
value: 45.635999999999996
|
|
- type: map_at_20
|
|
value: 45.265
|
|
- type: map_at_3
|
|
value: 41.703
|
|
- type: map_at_5
|
|
value: 43.488
|
|
- type: mrr_at_1
|
|
value: 33.160621761658035
|
|
- type: mrr_at_10
|
|
value: 44.697771883652734
|
|
- type: mrr_at_100
|
|
value: 45.59624815182174
|
|
- type: mrr_at_1000
|
|
value: 45.63609361771601
|
|
- type: mrr_at_20
|
|
value: 45.26480516767501
|
|
- type: mrr_at_3
|
|
value: 41.70264824409908
|
|
- type: mrr_at_5
|
|
value: 43.488054116292574
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 38.49809004106204
|
|
- type: nauc_map_at_1000_max
|
|
value: 31.640827883359986
|
|
- type: nauc_map_at_1000_std
|
|
value: 2.5944833693677563
|
|
- type: nauc_map_at_100_diff1
|
|
value: 38.47974017961114
|
|
- type: nauc_map_at_100_max
|
|
value: 31.6745580307424
|
|
- type: nauc_map_at_100_std
|
|
value: 2.6197669693649965
|
|
- type: nauc_map_at_10_diff1
|
|
value: 38.43029274269754
|
|
- type: nauc_map_at_10_max
|
|
value: 31.669351274164402
|
|
- type: nauc_map_at_10_std
|
|
value: 2.2938216424530955
|
|
- type: nauc_map_at_1_diff1
|
|
value: 42.39449280665502
|
|
- type: nauc_map_at_1_max
|
|
value: 27.396202491464315
|
|
- type: nauc_map_at_1_std
|
|
value: 0.39154393747181304
|
|
- type: nauc_map_at_20_diff1
|
|
value: 38.44710465218088
|
|
- type: nauc_map_at_20_max
|
|
value: 31.618626111686442
|
|
- type: nauc_map_at_20_std
|
|
value: 2.5092690901463994
|
|
- type: nauc_map_at_3_diff1
|
|
value: 38.68180058655341
|
|
- type: nauc_map_at_3_max
|
|
value: 30.48704606797293
|
|
- type: nauc_map_at_3_std
|
|
value: 1.6764325554613773
|
|
- type: nauc_map_at_5_diff1
|
|
value: 38.27528363570654
|
|
- type: nauc_map_at_5_max
|
|
value: 31.105696409714735
|
|
- type: nauc_map_at_5_std
|
|
value: 2.3132867223174043
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 38.49809004106204
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 31.640827883359986
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 2.5944833693677563
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 38.47974017961114
|
|
- type: nauc_mrr_at_100_max
|
|
value: 31.6745580307424
|
|
- type: nauc_mrr_at_100_std
|
|
value: 2.6197669693649965
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 38.43029274269754
|
|
- type: nauc_mrr_at_10_max
|
|
value: 31.669351274164402
|
|
- type: nauc_mrr_at_10_std
|
|
value: 2.2938216424530955
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 42.39449280665502
|
|
- type: nauc_mrr_at_1_max
|
|
value: 27.396202491464315
|
|
- type: nauc_mrr_at_1_std
|
|
value: 0.39154393747181304
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 38.44710465218088
|
|
- type: nauc_mrr_at_20_max
|
|
value: 31.618626111686442
|
|
- type: nauc_mrr_at_20_std
|
|
value: 2.5092690901463994
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 38.68180058655341
|
|
- type: nauc_mrr_at_3_max
|
|
value: 30.48704606797293
|
|
- type: nauc_mrr_at_3_std
|
|
value: 1.6764325554613773
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 38.27528363570654
|
|
- type: nauc_mrr_at_5_max
|
|
value: 31.105696409714735
|
|
- type: nauc_mrr_at_5_std
|
|
value: 2.3132867223174043
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 37.94639112622322
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 33.25000406312992
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 3.927246572224288
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 37.488139235799
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 34.38011697151766
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 4.94760159362139
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 37.318669958427996
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 34.19162673981376
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 3.2011892955083256
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 42.39449280665502
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 27.396202491464315
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 0.39154393747181304
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 37.290108058390985
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 34.108858641349556
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 4.169459504439506
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 37.62224828453568
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 31.519305313909218
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 2.087339522812091
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 36.888334499663785
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 32.6601407781184
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 3.2124484680546175
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 59.19282490825572
|
|
- type: nauc_precision_at_1000_max
|
|
value: 68.32089152822621
|
|
- type: nauc_precision_at_1000_std
|
|
value: 67.77161809421868
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 29.47575945272322
|
|
- type: nauc_precision_at_100_max
|
|
value: 63.42195725833949
|
|
- type: nauc_precision_at_100_std
|
|
value: 34.923105379547344
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 33.52501919318297
|
|
- type: nauc_precision_at_10_max
|
|
value: 44.49893440034256
|
|
- type: nauc_precision_at_10_std
|
|
value: 6.680426129369459
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 42.39449280665502
|
|
- type: nauc_precision_at_1_max
|
|
value: 27.396202491464315
|
|
- type: nauc_precision_at_1_std
|
|
value: 0.39154393747181304
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 32.17682672599943
|
|
- type: nauc_precision_at_20_max
|
|
value: 46.87049521936974
|
|
- type: nauc_precision_at_20_std
|
|
value: 13.53258332473726
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 34.54132207851944
|
|
- type: nauc_precision_at_3_max
|
|
value: 34.574775459010255
|
|
- type: nauc_precision_at_3_std
|
|
value: 3.298031208443393
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 32.475852196639195
|
|
- type: nauc_precision_at_5_max
|
|
value: 37.73978486643185
|
|
- type: nauc_precision_at_5_std
|
|
value: 6.185472179658329
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 59.19282490825427
|
|
- type: nauc_recall_at_1000_max
|
|
value: 68.32089152822542
|
|
- type: nauc_recall_at_1000_std
|
|
value: 67.77161809421989
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 29.475759452723388
|
|
- type: nauc_recall_at_100_max
|
|
value: 63.421957258339425
|
|
- type: nauc_recall_at_100_std
|
|
value: 34.92310537954746
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 33.525019193182956
|
|
- type: nauc_recall_at_10_max
|
|
value: 44.498934400342485
|
|
- type: nauc_recall_at_10_std
|
|
value: 6.680426129369434
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 42.39449280665502
|
|
- type: nauc_recall_at_1_max
|
|
value: 27.396202491464315
|
|
- type: nauc_recall_at_1_std
|
|
value: 0.39154393747181304
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 32.17682672599945
|
|
- type: nauc_recall_at_20_max
|
|
value: 46.87049521936974
|
|
- type: nauc_recall_at_20_std
|
|
value: 13.53258332473721
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 34.54132207851946
|
|
- type: nauc_recall_at_3_max
|
|
value: 34.5747754590102
|
|
- type: nauc_recall_at_3_std
|
|
value: 3.2980312084433936
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 32.47585219663912
|
|
- type: nauc_recall_at_5_max
|
|
value: 37.73978486643183
|
|
- type: nauc_recall_at_5_std
|
|
value: 6.18547217965832
|
|
- type: ndcg_at_1
|
|
value: 33.161
|
|
- type: ndcg_at_10
|
|
value: 50.638000000000005
|
|
- type: ndcg_at_100
|
|
value: 55.076
|
|
- type: ndcg_at_1000
|
|
value: 56.18300000000001
|
|
- type: ndcg_at_20
|
|
value: 52.681
|
|
- type: ndcg_at_3
|
|
value: 44.488
|
|
- type: ndcg_at_5
|
|
value: 47.705999999999996
|
|
- type: precision_at_1
|
|
value: 33.161
|
|
- type: precision_at_10
|
|
value: 6.9430000000000005
|
|
- type: precision_at_100
|
|
value: 0.9039999999999999
|
|
- type: precision_at_1000
|
|
value: 0.099
|
|
- type: precision_at_20
|
|
value: 3.873
|
|
- type: precision_at_3
|
|
value: 17.516000000000002
|
|
- type: precision_at_5
|
|
value: 12.073
|
|
- type: recall_at_1
|
|
value: 33.161
|
|
- type: recall_at_10
|
|
value: 69.43
|
|
- type: recall_at_100
|
|
value: 90.371
|
|
- type: recall_at_1000
|
|
value: 99.18
|
|
- type: recall_at_20
|
|
value: 77.461
|
|
- type: recall_at_3
|
|
value: 52.547
|
|
- type: recall_at_5
|
|
value: 60.363
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: fr
|
|
name: MTEB AmazonReviewsClassification (fr)
|
|
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
|
|
split: test
|
|
type: mteb/amazon_reviews_multi
|
|
metrics:
|
|
- type: accuracy
|
|
value: 52.622
|
|
- type: f1
|
|
value: 48.89589865194384
|
|
- type: f1_weighted
|
|
value: 48.89589865194384
|
|
- type: main_score
|
|
value: 52.622
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB BSARDRetrieval
|
|
revision: 5effa1b9b5fa3b0f9e12523e6e43e5f86a6e6d59
|
|
split: test
|
|
type: maastrichtlawtech/bsard
|
|
metrics:
|
|
- type: main_score
|
|
value: 59.458999999999996
|
|
- type: map_at_1
|
|
value: 8.559
|
|
- type: map_at_10
|
|
value: 15.853
|
|
- type: map_at_100
|
|
value: 16.955000000000002
|
|
- type: map_at_1000
|
|
value: 17.039
|
|
- type: map_at_20
|
|
value: 16.491
|
|
- type: map_at_3
|
|
value: 13.739
|
|
- type: map_at_5
|
|
value: 14.887
|
|
- type: mrr_at_1
|
|
value: 8.558558558558559
|
|
- type: mrr_at_10
|
|
value: 15.852995852995852
|
|
- type: mrr_at_100
|
|
value: 16.95536191852861
|
|
- type: mrr_at_1000
|
|
value: 17.03894008776081
|
|
- type: mrr_at_20
|
|
value: 16.490710101391212
|
|
- type: mrr_at_3
|
|
value: 13.738738738738734
|
|
- type: mrr_at_5
|
|
value: 14.887387387387385
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 22.6427616709538
|
|
- type: nauc_map_at_1000_max
|
|
value: 30.273021433334108
|
|
- type: nauc_map_at_1000_std
|
|
value: 8.648862859092157
|
|
- type: nauc_map_at_100_diff1
|
|
value: 22.523593314805954
|
|
- type: nauc_map_at_100_max
|
|
value: 30.197098780769366
|
|
- type: nauc_map_at_100_std
|
|
value: 8.638222954134465
|
|
- type: nauc_map_at_10_diff1
|
|
value: 22.382593376046035
|
|
- type: nauc_map_at_10_max
|
|
value: 30.80647774104949
|
|
- type: nauc_map_at_10_std
|
|
value: 7.6451773140303825
|
|
- type: nauc_map_at_1_diff1
|
|
value: 32.27835486300824
|
|
- type: nauc_map_at_1_max
|
|
value: 31.839925744574
|
|
- type: nauc_map_at_1_std
|
|
value: 7.524965617228806
|
|
- type: nauc_map_at_20_diff1
|
|
value: 22.78130766181537
|
|
- type: nauc_map_at_20_max
|
|
value: 30.207832515412452
|
|
- type: nauc_map_at_20_std
|
|
value: 7.988030006241385
|
|
- type: nauc_map_at_3_diff1
|
|
value: 21.54291029527254
|
|
- type: nauc_map_at_3_max
|
|
value: 30.60738044134162
|
|
- type: nauc_map_at_3_std
|
|
value: 5.115709517278456
|
|
- type: nauc_map_at_5_diff1
|
|
value: 22.891658440504543
|
|
- type: nauc_map_at_5_max
|
|
value: 30.40238430175482
|
|
- type: nauc_map_at_5_std
|
|
value: 6.496264144977426
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 22.6427616709538
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 30.273021433334108
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 8.648862859092157
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 22.523593314805954
|
|
- type: nauc_mrr_at_100_max
|
|
value: 30.197098780769366
|
|
- type: nauc_mrr_at_100_std
|
|
value: 8.638222954134465
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 22.382593376046035
|
|
- type: nauc_mrr_at_10_max
|
|
value: 30.80647774104949
|
|
- type: nauc_mrr_at_10_std
|
|
value: 7.6451773140303825
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 32.27835486300824
|
|
- type: nauc_mrr_at_1_max
|
|
value: 31.839925744574
|
|
- type: nauc_mrr_at_1_std
|
|
value: 7.524965617228806
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 22.78130766181537
|
|
- type: nauc_mrr_at_20_max
|
|
value: 30.207832515412452
|
|
- type: nauc_mrr_at_20_std
|
|
value: 7.988030006241385
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 21.54291029527254
|
|
- type: nauc_mrr_at_3_max
|
|
value: 30.60738044134162
|
|
- type: nauc_mrr_at_3_std
|
|
value: 5.115709517278456
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 22.891658440504543
|
|
- type: nauc_mrr_at_5_max
|
|
value: 30.40238430175482
|
|
- type: nauc_mrr_at_5_std
|
|
value: 6.496264144977426
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 22.131590111018863
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 30.119495176526417
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 14.152746889343884
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 19.59019307197614
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 29.26698074164439
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 14.64843229218199
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 20.04399986794229
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 30.370494010101606
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 9.076324266988427
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 32.27835486300824
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 31.839925744574
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 7.524965617228806
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 21.047174465558204
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 28.383850745017487
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 10.079085665060253
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 18.696202337264843
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 29.95559912145818
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 4.515594333379446
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 21.14710675076888
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 29.60877022537729
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 6.721635773882387
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 30.982325786968197
|
|
- type: nauc_precision_at_1000_max
|
|
value: 34.26481840304951
|
|
- type: nauc_precision_at_1000_std
|
|
value: 43.39003460634655
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 11.987279247967425
|
|
- type: nauc_precision_at_100_max
|
|
value: 28.50285582800895
|
|
- type: nauc_precision_at_100_std
|
|
value: 35.49648389671331
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 15.562900584507142
|
|
- type: nauc_precision_at_10_max
|
|
value: 29.558066061869663
|
|
- type: nauc_precision_at_10_std
|
|
value: 12.47595674036553
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 32.27835486300824
|
|
- type: nauc_precision_at_1_max
|
|
value: 31.839925744574
|
|
- type: nauc_precision_at_1_std
|
|
value: 7.524965617228806
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 18.081035071003427
|
|
- type: nauc_precision_at_20_max
|
|
value: 23.85063262716287
|
|
- type: nauc_precision_at_20_std
|
|
value: 15.071481920870877
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 12.597351208698534
|
|
- type: nauc_precision_at_3_max
|
|
value: 28.496818992459538
|
|
- type: nauc_precision_at_3_std
|
|
value: 3.2373330095471893
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 17.904830065631092
|
|
- type: nauc_precision_at_5_max
|
|
value: 27.89909851354525
|
|
- type: nauc_precision_at_5_std
|
|
value: 7.3432451499420734
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 30.982325786968097
|
|
- type: nauc_recall_at_1000_max
|
|
value: 34.264818403049496
|
|
- type: nauc_recall_at_1000_std
|
|
value: 43.39003460634647
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 11.987279247967388
|
|
- type: nauc_recall_at_100_max
|
|
value: 28.502855828008883
|
|
- type: nauc_recall_at_100_std
|
|
value: 35.49648389671325
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 15.562900584507085
|
|
- type: nauc_recall_at_10_max
|
|
value: 29.558066061869624
|
|
- type: nauc_recall_at_10_std
|
|
value: 12.475956740365447
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 32.27835486300824
|
|
- type: nauc_recall_at_1_max
|
|
value: 31.839925744574
|
|
- type: nauc_recall_at_1_std
|
|
value: 7.524965617228806
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 18.081035071003342
|
|
- type: nauc_recall_at_20_max
|
|
value: 23.850632627162785
|
|
- type: nauc_recall_at_20_std
|
|
value: 15.071481920870786
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 12.597351208698562
|
|
- type: nauc_recall_at_3_max
|
|
value: 28.496818992459545
|
|
- type: nauc_recall_at_3_std
|
|
value: 3.237333009547214
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 17.90483006563107
|
|
- type: nauc_recall_at_5_max
|
|
value: 27.89909851354522
|
|
- type: nauc_recall_at_5_std
|
|
value: 7.343245149942006
|
|
- type: ndcg_at_1
|
|
value: 8.559
|
|
- type: ndcg_at_10
|
|
value: 19.828000000000003
|
|
- type: ndcg_at_100
|
|
value: 25.468000000000004
|
|
- type: ndcg_at_1000
|
|
value: 28.058
|
|
- type: ndcg_at_20
|
|
value: 22.122
|
|
- type: ndcg_at_3
|
|
value: 15.524
|
|
- type: ndcg_at_5
|
|
value: 17.579
|
|
- type: precision_at_1
|
|
value: 8.559
|
|
- type: precision_at_10
|
|
value: 3.243
|
|
- type: precision_at_100
|
|
value: 0.5950000000000001
|
|
- type: precision_at_1000
|
|
value: 0.08099999999999999
|
|
- type: precision_at_20
|
|
value: 2.072
|
|
- type: precision_at_3
|
|
value: 6.907000000000001
|
|
- type: precision_at_5
|
|
value: 5.135
|
|
- type: recall_at_1
|
|
value: 8.559
|
|
- type: recall_at_10
|
|
value: 32.432
|
|
- type: recall_at_100
|
|
value: 59.458999999999996
|
|
- type: recall_at_1000
|
|
value: 80.631
|
|
- type: recall_at_20
|
|
value: 41.441
|
|
- type: recall_at_3
|
|
value: 20.721
|
|
- type: recall_at_5
|
|
value: 25.676
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB HALClusteringS2S
|
|
revision: e06ebbbb123f8144bef1a5d18796f3dec9ae2915
|
|
split: test
|
|
type: lyon-nlp/clustering-hal-s2s
|
|
metrics:
|
|
- type: main_score
|
|
value: 26.958035381361377
|
|
- type: v_measure
|
|
value: 26.958035381361377
|
|
- type: v_measure_std
|
|
value: 2.401353383071989
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: fr
|
|
name: MTEB MLSUMClusteringP2P (fr)
|
|
revision: b5d54f8f3b61ae17845046286940f03c6bc79bc7
|
|
split: test
|
|
type: reciTAL/mlsum
|
|
metrics:
|
|
- type: main_score
|
|
value: 46.15554988136895
|
|
- type: v_measure
|
|
value: 46.15554988136895
|
|
- type: v_measure_std
|
|
value: 2.459531525134688
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: fr
|
|
name: MTEB MLSUMClusteringS2S (fr)
|
|
revision: b5d54f8f3b61ae17845046286940f03c6bc79bc7
|
|
split: test
|
|
type: reciTAL/mlsum
|
|
metrics:
|
|
- type: main_score
|
|
value: 45.73187202144909
|
|
- type: v_measure
|
|
value: 45.73187202144909
|
|
- type: v_measure_std
|
|
value: 1.6402520163270633
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: fr
|
|
name: MTEB MTOPDomainClassification (fr)
|
|
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
|
|
split: test
|
|
type: mteb/mtop_domain
|
|
metrics:
|
|
- type: accuracy
|
|
value: 95.78766050735986
|
|
- type: f1
|
|
value: 95.61497706645892
|
|
- type: f1_weighted
|
|
value: 95.79887587161483
|
|
- type: main_score
|
|
value: 95.78766050735986
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: fr
|
|
name: MTEB MTOPIntentClassification (fr)
|
|
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
|
|
split: test
|
|
type: mteb/mtop_intent
|
|
metrics:
|
|
- type: accuracy
|
|
value: 80.8800501096148
|
|
- type: f1
|
|
value: 53.9945274705194
|
|
- type: f1_weighted
|
|
value: 80.94438738414857
|
|
- type: main_score
|
|
value: 80.8800501096148
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: fra
|
|
name: MTEB MasakhaNEWSClassification (fra)
|
|
revision: 18193f187b92da67168c655c9973a165ed9593dd
|
|
split: test
|
|
type: mteb/masakhanews
|
|
metrics:
|
|
- type: accuracy
|
|
value: 83.6255924170616
|
|
- type: f1
|
|
value: 79.70294641135138
|
|
- type: f1_weighted
|
|
value: 83.33457992982105
|
|
- type: main_score
|
|
value: 83.6255924170616
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: fra
|
|
name: MTEB MasakhaNEWSClusteringP2P (fra)
|
|
revision: 8ccc72e69e65f40c70e117d8b3c08306bb788b60
|
|
split: test
|
|
type: masakhane/masakhanews
|
|
metrics:
|
|
- type: main_score
|
|
value: 77.1970570860131
|
|
- type: v_measure
|
|
value: 77.1970570860131
|
|
- type: v_measure_std
|
|
value: 22.0055550035463
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: fra
|
|
name: MTEB MasakhaNEWSClusteringS2S (fra)
|
|
revision: 8ccc72e69e65f40c70e117d8b3c08306bb788b60
|
|
split: test
|
|
type: masakhane/masakhanews
|
|
metrics:
|
|
- type: main_score
|
|
value: 65.92601417312947
|
|
- type: v_measure
|
|
value: 65.92601417312947
|
|
- type: v_measure_std
|
|
value: 30.421071440935687
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: fr
|
|
name: MTEB MassiveIntentClassification (fr)
|
|
revision: 4672e20407010da34463acc759c162ca9734bca6
|
|
split: test
|
|
type: mteb/amazon_massive_intent
|
|
metrics:
|
|
- type: accuracy
|
|
value: 69.5359784801614
|
|
- type: f1
|
|
value: 64.640488940591
|
|
- type: f1_weighted
|
|
value: 67.85916565361048
|
|
- type: main_score
|
|
value: 69.5359784801614
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: fr
|
|
name: MTEB MassiveScenarioClassification (fr)
|
|
revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
|
|
split: test
|
|
type: mteb/amazon_massive_scenario
|
|
metrics:
|
|
- type: accuracy
|
|
value: 78.52723604572965
|
|
- type: f1
|
|
value: 77.1995224144067
|
|
- type: f1_weighted
|
|
value: 78.1215987283123
|
|
- type: main_score
|
|
value: 78.52723604572965
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: fr
|
|
name: MTEB MintakaRetrieval (fr)
|
|
revision: efa78cc2f74bbcd21eff2261f9e13aebe40b814e
|
|
split: test
|
|
type: jinaai/mintakaqa
|
|
metrics:
|
|
- type: main_score
|
|
value: 26.448
|
|
- type: map_at_1
|
|
value: 14.947
|
|
- type: map_at_10
|
|
value: 22.303
|
|
- type: map_at_100
|
|
value: 23.477999999999998
|
|
- type: map_at_1000
|
|
value: 23.586
|
|
- type: map_at_20
|
|
value: 22.962
|
|
- type: map_at_3
|
|
value: 19.949
|
|
- type: map_at_5
|
|
value: 21.252
|
|
- type: mrr_at_1
|
|
value: 14.946764946764945
|
|
- type: mrr_at_10
|
|
value: 22.303001053001033
|
|
- type: mrr_at_100
|
|
value: 23.478040499941816
|
|
- type: mrr_at_1000
|
|
value: 23.585987565381252
|
|
- type: mrr_at_20
|
|
value: 22.96198948271138
|
|
- type: mrr_at_3
|
|
value: 19.949494949494913
|
|
- type: mrr_at_5
|
|
value: 21.251706251706192
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 30.124123232611005
|
|
- type: nauc_map_at_1000_max
|
|
value: 19.329718056410893
|
|
- type: nauc_map_at_1000_std
|
|
value: 3.7304142418877606
|
|
- type: nauc_map_at_100_diff1
|
|
value: 30.06763654065989
|
|
- type: nauc_map_at_100_max
|
|
value: 19.339926348634375
|
|
- type: nauc_map_at_100_std
|
|
value: 3.7507886962889376
|
|
- type: nauc_map_at_10_diff1
|
|
value: 30.235621359267817
|
|
- type: nauc_map_at_10_max
|
|
value: 19.315231135265865
|
|
- type: nauc_map_at_10_std
|
|
value: 3.888262415552999
|
|
- type: nauc_map_at_1_diff1
|
|
value: 37.87356036243269
|
|
- type: nauc_map_at_1_max
|
|
value: 17.63892349776284
|
|
- type: nauc_map_at_1_std
|
|
value: -2.0575597858386208
|
|
- type: nauc_map_at_20_diff1
|
|
value: 30.06800385756772
|
|
- type: nauc_map_at_20_max
|
|
value: 19.172804564418264
|
|
- type: nauc_map_at_20_std
|
|
value: 3.721149536049358
|
|
- type: nauc_map_at_3_diff1
|
|
value: 32.09160567273595
|
|
- type: nauc_map_at_3_max
|
|
value: 19.055280691204825
|
|
- type: nauc_map_at_3_std
|
|
value: 1.9160849079572526
|
|
- type: nauc_map_at_5_diff1
|
|
value: 30.81034541116131
|
|
- type: nauc_map_at_5_max
|
|
value: 19.172166581396308
|
|
- type: nauc_map_at_5_std
|
|
value: 3.251197681984862
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 30.12412337741088
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 19.329717809214035
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 3.730414425912248
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 30.06763654065989
|
|
- type: nauc_mrr_at_100_max
|
|
value: 19.339926348634375
|
|
- type: nauc_mrr_at_100_std
|
|
value: 3.7507886962889376
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 30.235621359267817
|
|
- type: nauc_mrr_at_10_max
|
|
value: 19.315231135265865
|
|
- type: nauc_mrr_at_10_std
|
|
value: 3.888262415552999
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 37.87356036243269
|
|
- type: nauc_mrr_at_1_max
|
|
value: 17.63892349776284
|
|
- type: nauc_mrr_at_1_std
|
|
value: -2.0575597858386208
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 30.06800385756772
|
|
- type: nauc_mrr_at_20_max
|
|
value: 19.172804564418264
|
|
- type: nauc_mrr_at_20_std
|
|
value: 3.721149536049358
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 32.09160567273595
|
|
- type: nauc_mrr_at_3_max
|
|
value: 19.055280691204825
|
|
- type: nauc_mrr_at_3_std
|
|
value: 1.9160849079572526
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 30.81034541116131
|
|
- type: nauc_mrr_at_5_max
|
|
value: 19.172166581396308
|
|
- type: nauc_mrr_at_5_std
|
|
value: 3.251197681984862
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 28.057639637340476
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 20.172072747981893
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 5.991944827605241
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 26.60019642442434
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 20.47271103053784
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 6.489412476969333
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 27.165894912173762
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 19.79447862928707
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 6.648857204092722
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 37.87356036243269
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 17.63892349776284
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -2.0575597858386208
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 26.582793970516843
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 19.348538329936638
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 6.138040315782395
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 30.57338000196413
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 19.37852889877986
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 3.0568087546329408
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 28.469299405769632
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 19.599386892314122
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 5.299940395199246
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 24.170281200655943
|
|
- type: nauc_precision_at_1000_max
|
|
value: 39.623019898347664
|
|
- type: nauc_precision_at_1000_std
|
|
value: 44.81985014306762
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 14.474857644755179
|
|
- type: nauc_precision_at_100_max
|
|
value: 26.05636850160609
|
|
- type: nauc_precision_at_100_std
|
|
value: 16.53010919038197
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 19.584122367964167
|
|
- type: nauc_precision_at_10_max
|
|
value: 20.86686195708289
|
|
- type: nauc_precision_at_10_std
|
|
value: 13.525636908101404
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 37.87356036243269
|
|
- type: nauc_precision_at_1_max
|
|
value: 17.63892349776284
|
|
- type: nauc_precision_at_1_std
|
|
value: -2.0575597858386208
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 17.420915050615722
|
|
- type: nauc_precision_at_20_max
|
|
value: 19.45722509579383
|
|
- type: nauc_precision_at_20_std
|
|
value: 12.077196513907348
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 26.865120457860247
|
|
- type: nauc_precision_at_3_max
|
|
value: 20.154933241021354
|
|
- type: nauc_precision_at_3_std
|
|
value: 5.86927947299836
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 22.803351569123205
|
|
- type: nauc_precision_at_5_max
|
|
value: 20.623962388912666
|
|
- type: nauc_precision_at_5_std
|
|
value: 10.348629762758872
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 24.170281200656042
|
|
- type: nauc_recall_at_1000_max
|
|
value: 39.62301989834765
|
|
- type: nauc_recall_at_1000_std
|
|
value: 44.8198501430671
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 14.474857644755254
|
|
- type: nauc_recall_at_100_max
|
|
value: 26.056368501606116
|
|
- type: nauc_recall_at_100_std
|
|
value: 16.530109190381985
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 19.58412236796417
|
|
- type: nauc_recall_at_10_max
|
|
value: 20.866861957082875
|
|
- type: nauc_recall_at_10_std
|
|
value: 13.5256369081014
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 37.87356036243269
|
|
- type: nauc_recall_at_1_max
|
|
value: 17.63892349776284
|
|
- type: nauc_recall_at_1_std
|
|
value: -2.0575597858386208
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 17.420915050615708
|
|
- type: nauc_recall_at_20_max
|
|
value: 19.45722509579385
|
|
- type: nauc_recall_at_20_std
|
|
value: 12.077196513907353
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 26.865120457860243
|
|
- type: nauc_recall_at_3_max
|
|
value: 20.15493324102137
|
|
- type: nauc_recall_at_3_std
|
|
value: 5.869279472998389
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 22.803351569123215
|
|
- type: nauc_recall_at_5_max
|
|
value: 20.62396238891266
|
|
- type: nauc_recall_at_5_std
|
|
value: 10.348629762758849
|
|
- type: ndcg_at_1
|
|
value: 14.947
|
|
- type: ndcg_at_10
|
|
value: 26.448
|
|
- type: ndcg_at_100
|
|
value: 32.78
|
|
- type: ndcg_at_1000
|
|
value: 35.937000000000005
|
|
- type: ndcg_at_20
|
|
value: 28.842000000000002
|
|
- type: ndcg_at_3
|
|
value: 21.587999999999997
|
|
- type: ndcg_at_5
|
|
value: 23.942
|
|
- type: precision_at_1
|
|
value: 14.947
|
|
- type: precision_at_10
|
|
value: 3.972
|
|
- type: precision_at_100
|
|
value: 0.7080000000000001
|
|
- type: precision_at_1000
|
|
value: 0.096
|
|
- type: precision_at_20
|
|
value: 2.459
|
|
- type: precision_at_3
|
|
value: 8.777
|
|
- type: precision_at_5
|
|
value: 6.413
|
|
- type: recall_at_1
|
|
value: 14.947
|
|
- type: recall_at_10
|
|
value: 39.722
|
|
- type: recall_at_100
|
|
value: 70.844
|
|
- type: recall_at_1000
|
|
value: 96.274
|
|
- type: recall_at_20
|
|
value: 49.181000000000004
|
|
- type: recall_at_3
|
|
value: 26.331
|
|
- type: recall_at_5
|
|
value: 32.064
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: fr
|
|
name: MTEB OpusparcusPC (fr)
|
|
revision: 9e9b1f8ef51616073f47f306f7f47dd91663f86a
|
|
split: test
|
|
type: GEM/opusparcus
|
|
metrics:
|
|
- type: cosine_accuracy
|
|
value: 82.62942779291554
|
|
- type: cosine_accuracy_threshold
|
|
value: 83.4860622882843
|
|
- type: cosine_ap
|
|
value: 93.39616519364185
|
|
- type: cosine_f1
|
|
value: 88.03378695448146
|
|
- type: cosine_f1_threshold
|
|
value: 83.4860622882843
|
|
- type: cosine_precision
|
|
value: 83.45195729537367
|
|
- type: cosine_recall
|
|
value: 93.14796425024826
|
|
- type: dot_accuracy
|
|
value: 82.62942779291554
|
|
- type: dot_accuracy_threshold
|
|
value: 83.4860622882843
|
|
- type: dot_ap
|
|
value: 93.39616519364185
|
|
- type: dot_f1
|
|
value: 88.03378695448146
|
|
- type: dot_f1_threshold
|
|
value: 83.4860622882843
|
|
- type: dot_precision
|
|
value: 83.45195729537367
|
|
- type: dot_recall
|
|
value: 93.14796425024826
|
|
- type: euclidean_accuracy
|
|
value: 82.62942779291554
|
|
- type: euclidean_accuracy_threshold
|
|
value: 57.4698805809021
|
|
- type: euclidean_ap
|
|
value: 93.39616519364185
|
|
- type: euclidean_f1
|
|
value: 88.03378695448146
|
|
- type: euclidean_f1_threshold
|
|
value: 57.4698805809021
|
|
- type: euclidean_precision
|
|
value: 83.45195729537367
|
|
- type: euclidean_recall
|
|
value: 93.14796425024826
|
|
- type: main_score
|
|
value: 93.39616519364185
|
|
- type: manhattan_accuracy
|
|
value: 82.62942779291554
|
|
- type: manhattan_accuracy_threshold
|
|
value: 1306.7530632019043
|
|
- type: manhattan_ap
|
|
value: 93.34098710518775
|
|
- type: manhattan_f1
|
|
value: 87.78409090909089
|
|
- type: manhattan_f1_threshold
|
|
value: 1335.2685928344727
|
|
- type: manhattan_precision
|
|
value: 83.89140271493213
|
|
- type: manhattan_recall
|
|
value: 92.05561072492551
|
|
- type: max_ap
|
|
value: 93.39616519364185
|
|
- type: max_f1
|
|
value: 88.03378695448146
|
|
- type: max_precision
|
|
value: 83.89140271493213
|
|
- type: max_recall
|
|
value: 93.14796425024826
|
|
- type: similarity_accuracy
|
|
value: 82.62942779291554
|
|
- type: similarity_accuracy_threshold
|
|
value: 83.4860622882843
|
|
- type: similarity_ap
|
|
value: 93.39616519364185
|
|
- type: similarity_f1
|
|
value: 88.03378695448146
|
|
- type: similarity_f1_threshold
|
|
value: 83.4860622882843
|
|
- type: similarity_precision
|
|
value: 83.45195729537367
|
|
- type: similarity_recall
|
|
value: 93.14796425024826
|
|
task:
|
|
type: PairClassification
|
|
- dataset:
|
|
config: fr
|
|
name: MTEB PawsXPairClassification (fr)
|
|
revision: 8a04d940a42cd40658986fdd8e3da561533a3646
|
|
split: test
|
|
type: google-research-datasets/paws-x
|
|
metrics:
|
|
- type: cosine_accuracy
|
|
value: 60.8
|
|
- type: cosine_accuracy_threshold
|
|
value: 98.90193939208984
|
|
- type: cosine_ap
|
|
value: 60.50913122978733
|
|
- type: cosine_f1
|
|
value: 62.69411339833874
|
|
- type: cosine_f1_threshold
|
|
value: 95.17210125923157
|
|
- type: cosine_precision
|
|
value: 46.51661307609861
|
|
- type: cosine_recall
|
|
value: 96.12403100775194
|
|
- type: dot_accuracy
|
|
value: 60.8
|
|
- type: dot_accuracy_threshold
|
|
value: 98.9019513130188
|
|
- type: dot_ap
|
|
value: 60.49770725998639
|
|
- type: dot_f1
|
|
value: 62.69411339833874
|
|
- type: dot_f1_threshold
|
|
value: 95.17210721969604
|
|
- type: dot_precision
|
|
value: 46.51661307609861
|
|
- type: dot_recall
|
|
value: 96.12403100775194
|
|
- type: euclidean_accuracy
|
|
value: 60.8
|
|
- type: euclidean_accuracy_threshold
|
|
value: 14.819307625293732
|
|
- type: euclidean_ap
|
|
value: 60.50917425308617
|
|
- type: euclidean_f1
|
|
value: 62.69411339833874
|
|
- type: euclidean_f1_threshold
|
|
value: 31.07377290725708
|
|
- type: euclidean_precision
|
|
value: 46.51661307609861
|
|
- type: euclidean_recall
|
|
value: 96.12403100775194
|
|
- type: main_score
|
|
value: 60.73371250119265
|
|
- type: manhattan_accuracy
|
|
value: 60.9
|
|
- type: manhattan_accuracy_threshold
|
|
value: 354.8734188079834
|
|
- type: manhattan_ap
|
|
value: 60.73371250119265
|
|
- type: manhattan_f1
|
|
value: 62.70506744440393
|
|
- type: manhattan_f1_threshold
|
|
value: 711.578369140625
|
|
- type: manhattan_precision
|
|
value: 46.73913043478261
|
|
- type: manhattan_recall
|
|
value: 95.23809523809523
|
|
- type: max_ap
|
|
value: 60.73371250119265
|
|
- type: max_f1
|
|
value: 62.70506744440393
|
|
- type: max_precision
|
|
value: 46.73913043478261
|
|
- type: max_recall
|
|
value: 96.12403100775194
|
|
- type: similarity_accuracy
|
|
value: 60.8
|
|
- type: similarity_accuracy_threshold
|
|
value: 98.90193939208984
|
|
- type: similarity_ap
|
|
value: 60.50913122978733
|
|
- type: similarity_f1
|
|
value: 62.69411339833874
|
|
- type: similarity_f1_threshold
|
|
value: 95.17210125923157
|
|
- type: similarity_precision
|
|
value: 46.51661307609861
|
|
- type: similarity_recall
|
|
value: 96.12403100775194
|
|
task:
|
|
type: PairClassification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SICKFr
|
|
revision: e077ab4cf4774a1e36d86d593b150422fafd8e8a
|
|
split: test
|
|
type: Lajavaness/SICK-fr
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 81.02846310969592
|
|
- type: cosine_spearman
|
|
value: 77.47140335069184
|
|
- type: euclidean_pearson
|
|
value: 77.4818795209704
|
|
- type: euclidean_spearman
|
|
value: 77.4714043813526
|
|
- type: main_score
|
|
value: 77.47140335069184
|
|
- type: manhattan_pearson
|
|
value: 77.44622115854098
|
|
- type: manhattan_spearman
|
|
value: 77.29743297817558
|
|
- type: pearson
|
|
value: 81.02846310969592
|
|
- type: spearman
|
|
value: 77.47140335069184
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: fr
|
|
name: MTEB STS22 (fr)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 77.1356210910051
|
|
- type: cosine_spearman
|
|
value: 81.7065039306575
|
|
- type: euclidean_pearson
|
|
value: 79.32575551712296
|
|
- type: euclidean_spearman
|
|
value: 81.75624482168821
|
|
- type: main_score
|
|
value: 81.7065039306575
|
|
- type: manhattan_pearson
|
|
value: 81.05436417153798
|
|
- type: manhattan_spearman
|
|
value: 82.13370902176736
|
|
- type: pearson
|
|
value: 77.1356210910051
|
|
- type: spearman
|
|
value: 81.7065039306575
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: de-fr
|
|
name: MTEB STS22 (de-fr)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 61.40659325490285
|
|
- type: cosine_spearman
|
|
value: 64.21007088135842
|
|
- type: euclidean_pearson
|
|
value: 61.051174476106
|
|
- type: euclidean_spearman
|
|
value: 64.21007088135842
|
|
- type: main_score
|
|
value: 64.21007088135842
|
|
- type: manhattan_pearson
|
|
value: 60.225817072214525
|
|
- type: manhattan_spearman
|
|
value: 64.32288638294209
|
|
- type: pearson
|
|
value: 61.40659325490285
|
|
- type: spearman
|
|
value: 64.21007088135842
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: fr-pl
|
|
name: MTEB STS22 (fr-pl)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 88.17138238483673
|
|
- type: cosine_spearman
|
|
value: 84.51542547285167
|
|
- type: euclidean_pearson
|
|
value: 87.99782696047525
|
|
- type: euclidean_spearman
|
|
value: 84.51542547285167
|
|
- type: main_score
|
|
value: 84.51542547285167
|
|
- type: manhattan_pearson
|
|
value: 85.811937669563
|
|
- type: manhattan_spearman
|
|
value: 84.51542547285167
|
|
- type: pearson
|
|
value: 88.17138238483673
|
|
- type: spearman
|
|
value: 84.51542547285167
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: fr
|
|
name: MTEB STSBenchmarkMultilingualSTS (fr)
|
|
revision: 29afa2569dcedaaa2fe6a3dcfebab33d28b82e8c
|
|
split: test
|
|
type: mteb/stsb_multi_mt
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 79.98375089796882
|
|
- type: cosine_spearman
|
|
value: 81.06570417849169
|
|
- type: euclidean_pearson
|
|
value: 79.44759787417051
|
|
- type: euclidean_spearman
|
|
value: 81.06430479357311
|
|
- type: main_score
|
|
value: 81.06570417849169
|
|
- type: manhattan_pearson
|
|
value: 79.34683573713086
|
|
- type: manhattan_spearman
|
|
value: 81.00584846124926
|
|
- type: pearson
|
|
value: 79.98375089796882
|
|
- type: spearman
|
|
value: 81.06570417849169
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SummEvalFr
|
|
revision: b385812de6a9577b6f4d0f88c6a6e35395a94054
|
|
split: test
|
|
type: lyon-nlp/summarization-summeval-fr-p2p
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 31.198220154029464
|
|
- type: cosine_spearman
|
|
value: 30.886000528607877
|
|
- type: dot_pearson
|
|
value: 31.19822718500702
|
|
- type: dot_spearman
|
|
value: 30.86590068433314
|
|
- type: main_score
|
|
value: 30.886000528607877
|
|
- type: pearson
|
|
value: 31.198220154029464
|
|
- type: spearman
|
|
value: 30.886000528607877
|
|
task:
|
|
type: Summarization
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SyntecReranking
|
|
revision: daf0863838cd9e3ba50544cdce3ac2b338a1b0ad
|
|
split: test
|
|
type: lyon-nlp/mteb-fr-reranking-syntec-s2p
|
|
metrics:
|
|
- type: main_score
|
|
value: 86.6
|
|
- type: map
|
|
value: 86.6
|
|
- type: mrr
|
|
value: 86.6
|
|
- type: nAUC_map_diff1
|
|
value: 59.66160008216082
|
|
- type: nAUC_map_max
|
|
value: 19.768885092568734
|
|
- type: nAUC_map_std
|
|
value: 44.66975354255961
|
|
- type: nAUC_mrr_diff1
|
|
value: 59.66160008216082
|
|
- type: nAUC_mrr_max
|
|
value: 19.768885092568734
|
|
- type: nAUC_mrr_std
|
|
value: 44.66975354255961
|
|
task:
|
|
type: Reranking
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SyntecRetrieval
|
|
revision: 19661ccdca4dfc2d15122d776b61685f48c68ca9
|
|
split: test
|
|
type: lyon-nlp/mteb-fr-retrieval-syntec-s2p
|
|
metrics:
|
|
- type: main_score
|
|
value: 81.899
|
|
- type: map_at_1
|
|
value: 64.0
|
|
- type: map_at_10
|
|
value: 76.594
|
|
- type: map_at_100
|
|
value: 76.66199999999999
|
|
- type: map_at_1000
|
|
value: 76.66199999999999
|
|
- type: map_at_20
|
|
value: 76.644
|
|
- type: map_at_3
|
|
value: 74.833
|
|
- type: map_at_5
|
|
value: 76.183
|
|
- type: mrr_at_1
|
|
value: 64.0
|
|
- type: mrr_at_10
|
|
value: 76.59404761904761
|
|
- type: mrr_at_100
|
|
value: 76.66159147869675
|
|
- type: mrr_at_1000
|
|
value: 76.66159147869675
|
|
- type: mrr_at_20
|
|
value: 76.64404761904763
|
|
- type: mrr_at_3
|
|
value: 74.83333333333333
|
|
- type: mrr_at_5
|
|
value: 76.18333333333334
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 53.82627007182553
|
|
- type: nauc_map_at_1000_max
|
|
value: 17.927045359651704
|
|
- type: nauc_map_at_1000_std
|
|
value: -6.973071195715382
|
|
- type: nauc_map_at_100_diff1
|
|
value: 53.82627007182553
|
|
- type: nauc_map_at_100_max
|
|
value: 17.927045359651704
|
|
- type: nauc_map_at_100_std
|
|
value: -6.973071195715382
|
|
- type: nauc_map_at_10_diff1
|
|
value: 53.90625505629818
|
|
- type: nauc_map_at_10_max
|
|
value: 18.12979815440444
|
|
- type: nauc_map_at_10_std
|
|
value: -6.664265062780913
|
|
- type: nauc_map_at_1_diff1
|
|
value: 57.671797164388835
|
|
- type: nauc_map_at_1_max
|
|
value: 16.9354323668412
|
|
- type: nauc_map_at_1_std
|
|
value: -11.064631498275675
|
|
- type: nauc_map_at_20_diff1
|
|
value: 53.789271077104125
|
|
- type: nauc_map_at_20_max
|
|
value: 17.922015037605867
|
|
- type: nauc_map_at_20_std
|
|
value: -6.934974465544576
|
|
- type: nauc_map_at_3_diff1
|
|
value: 52.10054809507078
|
|
- type: nauc_map_at_3_max
|
|
value: 17.282564201023686
|
|
- type: nauc_map_at_3_std
|
|
value: -7.316507696153171
|
|
- type: nauc_map_at_5_diff1
|
|
value: 53.84305456072319
|
|
- type: nauc_map_at_5_max
|
|
value: 18.0761340059772
|
|
- type: nauc_map_at_5_std
|
|
value: -6.788097105243701
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 53.82627007182553
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 17.927045359651704
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -6.973071195715382
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 53.82627007182553
|
|
- type: nauc_mrr_at_100_max
|
|
value: 17.927045359651704
|
|
- type: nauc_mrr_at_100_std
|
|
value: -6.973071195715382
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 53.90625505629818
|
|
- type: nauc_mrr_at_10_max
|
|
value: 18.12979815440444
|
|
- type: nauc_mrr_at_10_std
|
|
value: -6.664265062780913
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 57.671797164388835
|
|
- type: nauc_mrr_at_1_max
|
|
value: 16.9354323668412
|
|
- type: nauc_mrr_at_1_std
|
|
value: -11.064631498275675
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 53.789271077104125
|
|
- type: nauc_mrr_at_20_max
|
|
value: 17.922015037605867
|
|
- type: nauc_mrr_at_20_std
|
|
value: -6.934974465544576
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 52.10054809507078
|
|
- type: nauc_mrr_at_3_max
|
|
value: 17.282564201023686
|
|
- type: nauc_mrr_at_3_std
|
|
value: -7.316507696153171
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 53.84305456072319
|
|
- type: nauc_mrr_at_5_max
|
|
value: 18.0761340059772
|
|
- type: nauc_mrr_at_5_std
|
|
value: -6.788097105243701
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 53.47773846493816
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 18.270810672735895
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -6.204392784046327
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 53.47773846493816
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 18.270810672735895
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -6.204392784046327
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 53.70897446254982
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 19.41340528944212
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -4.167245194562443
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 57.671797164388835
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 16.9354323668412
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -11.064631498275675
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 53.013882632385034
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 18.20334171980294
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -5.7313885736485455
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 49.798853568516044
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 17.88910440624622
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -5.959252175174665
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 53.565830685346896
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 19.301209293805627
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -4.5368156313357435
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: .nan
|
|
- type: nauc_precision_at_1000_max
|
|
value: .nan
|
|
- type: nauc_precision_at_1000_std
|
|
value: .nan
|
|
- type: nauc_precision_at_100_diff1
|
|
value: .nan
|
|
- type: nauc_precision_at_100_max
|
|
value: .nan
|
|
- type: nauc_precision_at_100_std
|
|
value: .nan
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 56.13912231559286
|
|
- type: nauc_precision_at_10_max
|
|
value: 56.13912231559286
|
|
- type: nauc_precision_at_10_std
|
|
value: 67.9038281979461
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 57.671797164388835
|
|
- type: nauc_precision_at_1_max
|
|
value: 16.9354323668412
|
|
- type: nauc_precision_at_1_std
|
|
value: -11.064631498275675
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 12.278244631185926
|
|
- type: nauc_precision_at_20_max
|
|
value: 12.278244631185926
|
|
- type: nauc_precision_at_20_std
|
|
value: 35.80765639589114
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 36.90404604415416
|
|
- type: nauc_precision_at_3_max
|
|
value: 21.58749248346349
|
|
- type: nauc_precision_at_3_std
|
|
value: 1.5204879305900956
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 53.47338935574264
|
|
- type: nauc_precision_at_5_max
|
|
value: 33.86554621848775
|
|
- type: nauc_precision_at_5_std
|
|
value: 22.00746965452886
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: .nan
|
|
- type: nauc_recall_at_1000_max
|
|
value: .nan
|
|
- type: nauc_recall_at_1000_std
|
|
value: .nan
|
|
- type: nauc_recall_at_100_diff1
|
|
value: .nan
|
|
- type: nauc_recall_at_100_max
|
|
value: .nan
|
|
- type: nauc_recall_at_100_std
|
|
value: .nan
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 56.13912231559305
|
|
- type: nauc_recall_at_10_max
|
|
value: 56.13912231559305
|
|
- type: nauc_recall_at_10_std
|
|
value: 67.903828197946
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 57.671797164388835
|
|
- type: nauc_recall_at_1_max
|
|
value: 16.9354323668412
|
|
- type: nauc_recall_at_1_std
|
|
value: -11.064631498275675
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 12.278244631185359
|
|
- type: nauc_recall_at_20_max
|
|
value: 12.278244631185359
|
|
- type: nauc_recall_at_20_std
|
|
value: 35.80765639589109
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 36.904046044154384
|
|
- type: nauc_recall_at_3_max
|
|
value: 21.587492483463492
|
|
- type: nauc_recall_at_3_std
|
|
value: 1.5204879305901602
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 53.47338935574226
|
|
- type: nauc_recall_at_5_max
|
|
value: 33.86554621848721
|
|
- type: nauc_recall_at_5_std
|
|
value: 22.00746965452852
|
|
- type: ndcg_at_1
|
|
value: 64.0
|
|
- type: ndcg_at_10
|
|
value: 81.899
|
|
- type: ndcg_at_100
|
|
value: 82.297
|
|
- type: ndcg_at_1000
|
|
value: 82.297
|
|
- type: ndcg_at_20
|
|
value: 82.126
|
|
- type: ndcg_at_3
|
|
value: 78.464
|
|
- type: ndcg_at_5
|
|
value: 80.917
|
|
- type: precision_at_1
|
|
value: 64.0
|
|
- type: precision_at_10
|
|
value: 9.8
|
|
- type: precision_at_100
|
|
value: 1.0
|
|
- type: precision_at_1000
|
|
value: 0.1
|
|
- type: precision_at_20
|
|
value: 4.95
|
|
- type: precision_at_3
|
|
value: 29.666999999999998
|
|
- type: precision_at_5
|
|
value: 19.0
|
|
- type: recall_at_1
|
|
value: 64.0
|
|
- type: recall_at_10
|
|
value: 98.0
|
|
- type: recall_at_100
|
|
value: 100.0
|
|
- type: recall_at_1000
|
|
value: 100.0
|
|
- type: recall_at_20
|
|
value: 99.0
|
|
- type: recall_at_3
|
|
value: 89.0
|
|
- type: recall_at_5
|
|
value: 95.0
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: fra-fra
|
|
name: MTEB XPQARetrieval (fr)
|
|
revision: c99d599f0a6ab9b85b065da6f9d94f9cf731679f
|
|
split: test
|
|
type: jinaai/xpqa
|
|
metrics:
|
|
- type: main_score
|
|
value: 62.09
|
|
- type: map_at_1
|
|
value: 36.073
|
|
- type: map_at_10
|
|
value: 55.677
|
|
- type: map_at_100
|
|
value: 57.118
|
|
- type: map_at_1000
|
|
value: 57.199
|
|
- type: map_at_20
|
|
value: 56.501999999999995
|
|
- type: map_at_3
|
|
value: 49.619
|
|
- type: map_at_5
|
|
value: 53.455
|
|
- type: mrr_at_1
|
|
value: 57.543391188251
|
|
- type: mrr_at_10
|
|
value: 65.30018861127007
|
|
- type: mrr_at_100
|
|
value: 65.94099315822325
|
|
- type: mrr_at_1000
|
|
value: 65.96453864135188
|
|
- type: mrr_at_20
|
|
value: 65.71224590825028
|
|
- type: mrr_at_3
|
|
value: 63.351134846461946
|
|
- type: mrr_at_5
|
|
value: 64.42590120160212
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 50.58271523935874
|
|
- type: nauc_map_at_1000_max
|
|
value: 43.13816564852953
|
|
- type: nauc_map_at_1000_std
|
|
value: 0.1844114463791253
|
|
- type: nauc_map_at_100_diff1
|
|
value: 50.55402514007517
|
|
- type: nauc_map_at_100_max
|
|
value: 43.131135237384484
|
|
- type: nauc_map_at_100_std
|
|
value: 0.1962985407010382
|
|
- type: nauc_map_at_10_diff1
|
|
value: 50.211948710332386
|
|
- type: nauc_map_at_10_max
|
|
value: 42.56586858276775
|
|
- type: nauc_map_at_10_std
|
|
value: -0.21682461914908613
|
|
- type: nauc_map_at_1_diff1
|
|
value: 58.97288229253611
|
|
- type: nauc_map_at_1_max
|
|
value: 27.09256578748547
|
|
- type: nauc_map_at_1_std
|
|
value: -3.128360909366587
|
|
- type: nauc_map_at_20_diff1
|
|
value: 50.33687763446524
|
|
- type: nauc_map_at_20_max
|
|
value: 42.89316787999387
|
|
- type: nauc_map_at_20_std
|
|
value: -0.2224194056336769
|
|
- type: nauc_map_at_3_diff1
|
|
value: 51.23147801843447
|
|
- type: nauc_map_at_3_max
|
|
value: 37.22691523640508
|
|
- type: nauc_map_at_3_std
|
|
value: -1.4704387784247346
|
|
- type: nauc_map_at_5_diff1
|
|
value: 50.66157676518992
|
|
- type: nauc_map_at_5_max
|
|
value: 41.58957149577394
|
|
- type: nauc_map_at_5_std
|
|
value: 0.16909716462753255
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 58.88383847534171
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 49.3245365217643
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 2.1575868813952894
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 58.885865820137276
|
|
- type: nauc_mrr_at_100_max
|
|
value: 49.32954909327622
|
|
- type: nauc_mrr_at_100_std
|
|
value: 2.1750885487117024
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 58.83761987066026
|
|
- type: nauc_mrr_at_10_max
|
|
value: 49.37803355533766
|
|
- type: nauc_mrr_at_10_std
|
|
value: 1.927477967317313
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 60.897823384674496
|
|
- type: nauc_mrr_at_1_max
|
|
value: 48.79303626218842
|
|
- type: nauc_mrr_at_1_std
|
|
value: 3.68732973455558
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 58.80334636154898
|
|
- type: nauc_mrr_at_20_max
|
|
value: 49.299926776474535
|
|
- type: nauc_mrr_at_20_std
|
|
value: 1.9599488796786029
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 59.21037240205004
|
|
- type: nauc_mrr_at_3_max
|
|
value: 49.14597672580709
|
|
- type: nauc_mrr_at_3_std
|
|
value: 1.0051764061328385
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 58.98849095570841
|
|
- type: nauc_mrr_at_5_max
|
|
value: 49.68364568027881
|
|
- type: nauc_mrr_at_5_std
|
|
value: 2.4739579029654366
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 52.31164533549997
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 45.69420989458311
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 1.1608489877596142
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 51.87286842964108
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 45.685834956792895
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 1.8157949218428466
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 50.57331251457611
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 44.44795063905562
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -0.3915488786381922
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 60.897823384674496
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 48.79303626218842
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 3.68732973455558
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 50.76487704699518
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 44.79388134049559
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -0.4213693889586553
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 51.177774035828605
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 43.73405047316084
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -1.18104282095782
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 51.15375930024702
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 43.7940523142017
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 0.8224796779269716
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -13.700846719394837
|
|
- type: nauc_precision_at_1000_max
|
|
value: 15.005182092410575
|
|
- type: nauc_precision_at_1000_std
|
|
value: 6.913901876028514
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -8.919890455110265
|
|
- type: nauc_precision_at_100_max
|
|
value: 20.85944528699816
|
|
- type: nauc_precision_at_100_std
|
|
value: 8.934660613911344
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 2.0626021976371662
|
|
- type: nauc_precision_at_10_max
|
|
value: 30.851331908454423
|
|
- type: nauc_precision_at_10_std
|
|
value: 4.512923316711585
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 60.897823384674496
|
|
- type: nauc_precision_at_1_max
|
|
value: 48.79303626218842
|
|
- type: nauc_precision_at_1_std
|
|
value: 3.68732973455558
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -1.9918582602200585
|
|
- type: nauc_precision_at_20_max
|
|
value: 27.779932491338315
|
|
- type: nauc_precision_at_20_std
|
|
value: 4.734186088720616
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 14.5090169489911
|
|
- type: nauc_precision_at_3_max
|
|
value: 37.59006778251299
|
|
- type: nauc_precision_at_3_std
|
|
value: 3.677659738072369
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 7.705804886616575
|
|
- type: nauc_precision_at_5_max
|
|
value: 36.0216894270471
|
|
- type: nauc_precision_at_5_std
|
|
value: 6.513474617464925
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 20.71811619738829
|
|
- type: nauc_recall_at_1000_max
|
|
value: 23.217180195398225
|
|
- type: nauc_recall_at_1000_std
|
|
value: 26.037508089878237
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 38.44958378050671
|
|
- type: nauc_recall_at_100_max
|
|
value: 40.99327582118083
|
|
- type: nauc_recall_at_100_std
|
|
value: 16.36015422588489
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 40.027789080211576
|
|
- type: nauc_recall_at_10_max
|
|
value: 38.82613587358396
|
|
- type: nauc_recall_at_10_std
|
|
value: -3.5237192778606596
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 58.97288229253611
|
|
- type: nauc_recall_at_1_max
|
|
value: 27.09256578748547
|
|
- type: nauc_recall_at_1_std
|
|
value: -3.128360909366587
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 37.818919303571406
|
|
- type: nauc_recall_at_20_max
|
|
value: 37.42703966259237
|
|
- type: nauc_recall_at_20_std
|
|
value: -4.770317748130178
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 45.13163472734054
|
|
- type: nauc_recall_at_3_max
|
|
value: 33.72267598718042
|
|
- type: nauc_recall_at_3_std
|
|
value: -4.443802840190085
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 43.05114612174671
|
|
- type: nauc_recall_at_5_max
|
|
value: 39.10347802906311
|
|
- type: nauc_recall_at_5_std
|
|
value: 0.4813526343602913
|
|
- type: ndcg_at_1
|
|
value: 57.543
|
|
- type: ndcg_at_10
|
|
value: 62.09
|
|
- type: ndcg_at_100
|
|
value: 67.216
|
|
- type: ndcg_at_1000
|
|
value: 68.60000000000001
|
|
- type: ndcg_at_20
|
|
value: 64.20700000000001
|
|
- type: ndcg_at_3
|
|
value: 56.952999999999996
|
|
- type: ndcg_at_5
|
|
value: 58.631
|
|
- type: precision_at_1
|
|
value: 57.543
|
|
- type: precision_at_10
|
|
value: 14.499
|
|
- type: precision_at_100
|
|
value: 1.8739999999999999
|
|
- type: precision_at_1000
|
|
value: 0.20600000000000002
|
|
- type: precision_at_20
|
|
value: 7.971
|
|
- type: precision_at_3
|
|
value: 34.446
|
|
- type: precision_at_5
|
|
value: 24.993000000000002
|
|
- type: recall_at_1
|
|
value: 36.073
|
|
- type: recall_at_10
|
|
value: 70.532
|
|
- type: recall_at_100
|
|
value: 90.63600000000001
|
|
- type: recall_at_1000
|
|
value: 99.577
|
|
- type: recall_at_20
|
|
value: 77.388
|
|
- type: recall_at_3
|
|
value: 54.786
|
|
- type: recall_at_5
|
|
value: 62.365
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: eng-fra
|
|
name: MTEB XPQARetrieval (eng-fra)
|
|
revision: c99d599f0a6ab9b85b065da6f9d94f9cf731679f
|
|
split: test
|
|
type: jinaai/xpqa
|
|
metrics:
|
|
- type: main_score
|
|
value: 34.795
|
|
- type: map_at_1
|
|
value: 13.818
|
|
- type: map_at_10
|
|
value: 28.221
|
|
- type: map_at_100
|
|
value: 30.715999999999998
|
|
- type: map_at_1000
|
|
value: 30.86
|
|
- type: map_at_20
|
|
value: 29.601
|
|
- type: map_at_3
|
|
value: 23.194
|
|
- type: map_at_5
|
|
value: 26.057999999999996
|
|
- type: mrr_at_1
|
|
value: 27.236315086782376
|
|
- type: mrr_at_10
|
|
value: 36.39890224002375
|
|
- type: mrr_at_100
|
|
value: 37.73446796439471
|
|
- type: mrr_at_1000
|
|
value: 37.79021013088287
|
|
- type: mrr_at_20
|
|
value: 37.17175635350331
|
|
- type: mrr_at_3
|
|
value: 33.73386737872722
|
|
- type: mrr_at_5
|
|
value: 35.24922118380064
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 32.30220782845437
|
|
- type: nauc_map_at_1000_max
|
|
value: 39.87665001530303
|
|
- type: nauc_map_at_1000_std
|
|
value: 5.7695221727058055
|
|
- type: nauc_map_at_100_diff1
|
|
value: 32.2694600306288
|
|
- type: nauc_map_at_100_max
|
|
value: 39.8855550981263
|
|
- type: nauc_map_at_100_std
|
|
value: 5.776881467271089
|
|
- type: nauc_map_at_10_diff1
|
|
value: 31.804039669931537
|
|
- type: nauc_map_at_10_max
|
|
value: 39.311719475930005
|
|
- type: nauc_map_at_10_std
|
|
value: 4.733050895784147
|
|
- type: nauc_map_at_1_diff1
|
|
value: 37.6388258626095
|
|
- type: nauc_map_at_1_max
|
|
value: 25.192760889147102
|
|
- type: nauc_map_at_1_std
|
|
value: 0.84012346712856
|
|
- type: nauc_map_at_20_diff1
|
|
value: 31.970587565845022
|
|
- type: nauc_map_at_20_max
|
|
value: 39.68812698793437
|
|
- type: nauc_map_at_20_std
|
|
value: 5.466710545588436
|
|
- type: nauc_map_at_3_diff1
|
|
value: 33.9083845702625
|
|
- type: nauc_map_at_3_max
|
|
value: 35.88443788757562
|
|
- type: nauc_map_at_3_std
|
|
value: 2.956590608487331
|
|
- type: nauc_map_at_5_diff1
|
|
value: 32.97116962607063
|
|
- type: nauc_map_at_5_max
|
|
value: 38.37964967819906
|
|
- type: nauc_map_at_5_std
|
|
value: 4.573297881379916
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 32.44126725520097
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 39.246349656799985
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 7.779126542191918
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 32.416259735518885
|
|
- type: nauc_mrr_at_100_max
|
|
value: 39.258316221246965
|
|
- type: nauc_mrr_at_100_std
|
|
value: 7.7944505673136
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 32.06932803779604
|
|
- type: nauc_mrr_at_10_max
|
|
value: 39.04853109147614
|
|
- type: nauc_mrr_at_10_std
|
|
value: 7.4138965935269505
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 37.27173489316227
|
|
- type: nauc_mrr_at_1_max
|
|
value: 40.3355905491979
|
|
- type: nauc_mrr_at_1_std
|
|
value: 6.933728079474825
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 32.329474296004804
|
|
- type: nauc_mrr_at_20_max
|
|
value: 39.0764118546337
|
|
- type: nauc_mrr_at_20_std
|
|
value: 7.696441003623004
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 33.08673004752433
|
|
- type: nauc_mrr_at_3_max
|
|
value: 39.95985433324281
|
|
- type: nauc_mrr_at_3_std
|
|
value: 7.62764311528151
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 32.818965514653684
|
|
- type: nauc_mrr_at_5_max
|
|
value: 39.34493265770003
|
|
- type: nauc_mrr_at_5_std
|
|
value: 7.778531920242
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 31.02864530261756
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 39.64187241406462
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 7.768459120817835
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 30.39095044516521
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 39.956877555291406
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 8.813305671545828
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 29.09294115578835
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 38.59245602933513
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 5.516145701680656
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 37.27173489316227
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 40.3355905491979
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 6.933728079474825
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 29.725541536865684
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 39.12781667827556
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 7.464557759930056
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 32.2472918241563
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 39.38528978160266
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 5.126228097274878
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 31.39000117667687
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 38.16838826710958
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 5.747613838798295
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 3.926032418467635
|
|
- type: nauc_precision_at_1000_max
|
|
value: 19.08045437036499
|
|
- type: nauc_precision_at_1000_std
|
|
value: 6.796129044597931
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 9.73958477384916
|
|
- type: nauc_precision_at_100_max
|
|
value: 29.07096859484853
|
|
- type: nauc_precision_at_100_std
|
|
value: 12.96991105140292
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 17.1980255233314
|
|
- type: nauc_precision_at_10_max
|
|
value: 43.52273606745023
|
|
- type: nauc_precision_at_10_std
|
|
value: 10.958034153583304
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 37.27173489316227
|
|
- type: nauc_precision_at_1_max
|
|
value: 40.3355905491979
|
|
- type: nauc_precision_at_1_std
|
|
value: 6.933728079474825
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 15.369446454490415
|
|
- type: nauc_precision_at_20_max
|
|
value: 39.48680497589929
|
|
- type: nauc_precision_at_20_std
|
|
value: 12.929898425260022
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 23.95767667939835
|
|
- type: nauc_precision_at_3_max
|
|
value: 45.09931497087944
|
|
- type: nauc_precision_at_3_std
|
|
value: 8.770453872723321
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 21.065461642416665
|
|
- type: nauc_precision_at_5_max
|
|
value: 44.72202962458711
|
|
- type: nauc_precision_at_5_std
|
|
value: 10.750776410192397
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 6.039018739578919
|
|
- type: nauc_recall_at_1000_max
|
|
value: 11.436015450640827
|
|
- type: nauc_recall_at_1000_std
|
|
value: 66.07591419148011
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 18.65963295269777
|
|
- type: nauc_recall_at_100_max
|
|
value: 38.12793731004431
|
|
- type: nauc_recall_at_100_std
|
|
value: 22.877750142093177
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 19.342831730746934
|
|
- type: nauc_recall_at_10_max
|
|
value: 31.63232417200137
|
|
- type: nauc_recall_at_10_std
|
|
value: 3.271699563530961
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 37.6388258626095
|
|
- type: nauc_recall_at_1_max
|
|
value: 25.192760889147102
|
|
- type: nauc_recall_at_1_std
|
|
value: 0.84012346712856
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 20.61391084945006
|
|
- type: nauc_recall_at_20_max
|
|
value: 32.5842740511191
|
|
- type: nauc_recall_at_20_std
|
|
value: 9.819500541742485
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 28.687240532045937
|
|
- type: nauc_recall_at_3_max
|
|
value: 31.72988302079546
|
|
- type: nauc_recall_at_3_std
|
|
value: 2.7062500297360295
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 25.62354289856022
|
|
- type: nauc_recall_at_5_max
|
|
value: 32.13732981730723
|
|
- type: nauc_recall_at_5_std
|
|
value: 4.661623958812741
|
|
- type: ndcg_at_1
|
|
value: 27.236
|
|
- type: ndcg_at_10
|
|
value: 34.795
|
|
- type: ndcg_at_100
|
|
value: 44.352000000000004
|
|
- type: ndcg_at_1000
|
|
value: 46.98
|
|
- type: ndcg_at_20
|
|
value: 38.537
|
|
- type: ndcg_at_3
|
|
value: 29.448
|
|
- type: ndcg_at_5
|
|
value: 30.996000000000002
|
|
- type: precision_at_1
|
|
value: 27.236
|
|
- type: precision_at_10
|
|
value: 10.427
|
|
- type: precision_at_100
|
|
value: 1.8769999999999998
|
|
- type: precision_at_1000
|
|
value: 0.22300000000000003
|
|
- type: precision_at_20
|
|
value: 6.589
|
|
- type: precision_at_3
|
|
value: 21.584
|
|
- type: precision_at_5
|
|
value: 16.555
|
|
- type: recall_at_1
|
|
value: 13.818
|
|
- type: recall_at_10
|
|
value: 44.826
|
|
- type: recall_at_100
|
|
value: 82.047
|
|
- type: recall_at_1000
|
|
value: 99.286
|
|
- type: recall_at_20
|
|
value: 56.615
|
|
- type: recall_at_3
|
|
value: 28.509
|
|
- type: recall_at_5
|
|
value: 35.472
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: fra-eng
|
|
name: MTEB XPQARetrieval (fra-eng)
|
|
revision: c99d599f0a6ab9b85b065da6f9d94f9cf731679f
|
|
split: test
|
|
type: jinaai/xpqa
|
|
metrics:
|
|
- type: main_score
|
|
value: 55.165
|
|
- type: map_at_1
|
|
value: 30.070999999999998
|
|
- type: map_at_10
|
|
value: 48.391
|
|
- type: map_at_100
|
|
value: 50.077000000000005
|
|
- type: map_at_1000
|
|
value: 50.175000000000004
|
|
- type: map_at_20
|
|
value: 49.425999999999995
|
|
- type: map_at_3
|
|
value: 43.108999999999995
|
|
- type: map_at_5
|
|
value: 46.331
|
|
- type: mrr_at_1
|
|
value: 47.79706275033378
|
|
- type: mrr_at_10
|
|
value: 57.112499205289545
|
|
- type: mrr_at_100
|
|
value: 57.77653857554601
|
|
- type: mrr_at_1000
|
|
value: 57.810309267669155
|
|
- type: mrr_at_20
|
|
value: 57.50639791688323
|
|
- type: mrr_at_3
|
|
value: 55.028927458833955
|
|
- type: mrr_at_5
|
|
value: 56.19715175789934
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 48.30744877651571
|
|
- type: nauc_map_at_1000_max
|
|
value: 41.304811375830106
|
|
- type: nauc_map_at_1000_std
|
|
value: -3.319640191562977
|
|
- type: nauc_map_at_100_diff1
|
|
value: 48.24792131158136
|
|
- type: nauc_map_at_100_max
|
|
value: 41.289809346155096
|
|
- type: nauc_map_at_100_std
|
|
value: -3.3322490840754044
|
|
- type: nauc_map_at_10_diff1
|
|
value: 48.51735869545944
|
|
- type: nauc_map_at_10_max
|
|
value: 41.39091467858207
|
|
- type: nauc_map_at_10_std
|
|
value: -3.816834529081366
|
|
- type: nauc_map_at_1_diff1
|
|
value: 55.20976873300869
|
|
- type: nauc_map_at_1_max
|
|
value: 29.31564406699001
|
|
- type: nauc_map_at_1_std
|
|
value: -3.531295202744916
|
|
- type: nauc_map_at_20_diff1
|
|
value: 48.15994357210226
|
|
- type: nauc_map_at_20_max
|
|
value: 41.27059573974859
|
|
- type: nauc_map_at_20_std
|
|
value: -3.553567850461392
|
|
- type: nauc_map_at_3_diff1
|
|
value: 49.07638331745524
|
|
- type: nauc_map_at_3_max
|
|
value: 37.4344180429373
|
|
- type: nauc_map_at_3_std
|
|
value: -4.793107974869855
|
|
- type: nauc_map_at_5_diff1
|
|
value: 48.610911544450566
|
|
- type: nauc_map_at_5_max
|
|
value: 40.36936409939194
|
|
- type: nauc_map_at_5_std
|
|
value: -4.494930285823858
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 54.361185624681966
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 44.38223734909631
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -4.6407251183091045
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 54.3534593950135
|
|
- type: nauc_mrr_at_100_max
|
|
value: 44.378173894610114
|
|
- type: nauc_mrr_at_100_std
|
|
value: -4.625111682775984
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 54.24312662266002
|
|
- type: nauc_mrr_at_10_max
|
|
value: 44.30746970923868
|
|
- type: nauc_mrr_at_10_std
|
|
value: -4.965217414238369
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 58.26954724773496
|
|
- type: nauc_mrr_at_1_max
|
|
value: 45.03422518009353
|
|
- type: nauc_mrr_at_1_std
|
|
value: -4.069334933239831
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 54.25997769977666
|
|
- type: nauc_mrr_at_20_max
|
|
value: 44.38402056799441
|
|
- type: nauc_mrr_at_20_std
|
|
value: -4.671395366726689
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 54.499361492963985
|
|
- type: nauc_mrr_at_3_max
|
|
value: 43.81936137776164
|
|
- type: nauc_mrr_at_3_std
|
|
value: -5.644776625702544
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 54.44957576550037
|
|
- type: nauc_mrr_at_5_max
|
|
value: 43.983826735470124
|
|
- type: nauc_mrr_at_5_std
|
|
value: -5.796801921865972
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 49.15241156513385
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 42.45980120922761
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -2.3353260193872605
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 48.24122686676774
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 42.27679493782058
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -1.5752369584570114
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 48.5509813605824
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 42.59298249833255
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -3.672669315491546
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 58.26954724773496
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 45.03422518009353
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -4.069334933239831
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 47.729261088005316
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 42.49497033902468
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -2.6719433358977773
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 48.68223689824344
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 40.9157048148036
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -5.637336437839516
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 48.69726991107552
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 41.152294520697076
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -5.48123275220102
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -10.425039324403782
|
|
- type: nauc_precision_at_1000_max
|
|
value: 7.051352071885475
|
|
- type: nauc_precision_at_1000_std
|
|
value: 4.456043136940008
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -6.528489272274514
|
|
- type: nauc_precision_at_100_max
|
|
value: 12.611149343017736
|
|
- type: nauc_precision_at_100_std
|
|
value: 5.918229501417929
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 9.37469315859335
|
|
- type: nauc_precision_at_10_max
|
|
value: 29.792160957981938
|
|
- type: nauc_precision_at_10_std
|
|
value: 0.2316309488416353
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 58.26954724773496
|
|
- type: nauc_precision_at_1_max
|
|
value: 45.03422518009353
|
|
- type: nauc_precision_at_1_std
|
|
value: -4.069334933239831
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 2.981751622851337
|
|
- type: nauc_precision_at_20_max
|
|
value: 23.312084195651227
|
|
- type: nauc_precision_at_20_std
|
|
value: 2.560521133286893
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 20.831474725533468
|
|
- type: nauc_precision_at_3_max
|
|
value: 34.732843194059996
|
|
- type: nauc_precision_at_3_std
|
|
value: -3.379064346220114
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 14.628778037588857
|
|
- type: nauc_precision_at_5_max
|
|
value: 33.5567398421705
|
|
- type: nauc_precision_at_5_std
|
|
value: -2.4525869923256236
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 24.629562614981076
|
|
- type: nauc_recall_at_1000_max
|
|
value: 37.74776159843809
|
|
- type: nauc_recall_at_1000_std
|
|
value: 45.84365921167674
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 28.656294603430176
|
|
- type: nauc_recall_at_100_max
|
|
value: 34.99333512037935
|
|
- type: nauc_recall_at_100_std
|
|
value: 18.07167333451945
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 39.579271628779686
|
|
- type: nauc_recall_at_10_max
|
|
value: 39.65055294313406
|
|
- type: nauc_recall_at_10_std
|
|
value: -1.4953189564586904
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 55.20976873300869
|
|
- type: nauc_recall_at_1_max
|
|
value: 29.31564406699001
|
|
- type: nauc_recall_at_1_std
|
|
value: -3.531295202744916
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 35.59952531108398
|
|
- type: nauc_recall_at_20_max
|
|
value: 39.735665662589234
|
|
- type: nauc_recall_at_20_std
|
|
value: 2.746812413081314
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 42.180790443876234
|
|
- type: nauc_recall_at_3_max
|
|
value: 33.23529070499019
|
|
- type: nauc_recall_at_3_std
|
|
value: -7.102867270573987
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 41.34875509720362
|
|
- type: nauc_recall_at_5_max
|
|
value: 36.67737500141328
|
|
- type: nauc_recall_at_5_std
|
|
value: -7.16711230678949
|
|
- type: ndcg_at_1
|
|
value: 47.797
|
|
- type: ndcg_at_10
|
|
value: 55.165
|
|
- type: ndcg_at_100
|
|
value: 61.072
|
|
- type: ndcg_at_1000
|
|
value: 62.766999999999996
|
|
- type: ndcg_at_20
|
|
value: 57.603
|
|
- type: ndcg_at_3
|
|
value: 50.134
|
|
- type: ndcg_at_5
|
|
value: 51.711
|
|
- type: precision_at_1
|
|
value: 47.797
|
|
- type: precision_at_10
|
|
value: 13.150999999999998
|
|
- type: precision_at_100
|
|
value: 1.8370000000000002
|
|
- type: precision_at_1000
|
|
value: 0.20600000000000002
|
|
- type: precision_at_20
|
|
value: 7.517
|
|
- type: precision_at_3
|
|
value: 30.975
|
|
- type: precision_at_5
|
|
value: 22.27
|
|
- type: recall_at_1
|
|
value: 30.070999999999998
|
|
- type: recall_at_10
|
|
value: 65.352
|
|
- type: recall_at_100
|
|
value: 88.31099999999999
|
|
- type: recall_at_1000
|
|
value: 99.417
|
|
- type: recall_at_20
|
|
value: 72.65
|
|
- type: recall_at_3
|
|
value: 49.891000000000005
|
|
- type: recall_at_5
|
|
value: 56.949000000000005
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB AllegroReviews
|
|
revision: b89853e6de927b0e3bfa8ecc0e56fe4e02ceafc6
|
|
split: test
|
|
type: PL-MTEB/allegro-reviews
|
|
metrics:
|
|
- type: accuracy
|
|
value: 53.48906560636182
|
|
- type: f1
|
|
value: 41.948000361532074
|
|
- type: f1_weighted
|
|
value: 50.64284561538599
|
|
- type: main_score
|
|
value: 53.48906560636182
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB ArguAna-PL
|
|
revision: 63fc86750af76253e8c760fc9e534bbf24d260a2
|
|
split: test
|
|
type: clarin-knext/arguana-pl
|
|
metrics:
|
|
- type: main_score
|
|
value: 49.913000000000004
|
|
- type: map_at_1
|
|
value: 24.04
|
|
- type: map_at_10
|
|
value: 40.493
|
|
- type: map_at_100
|
|
value: 41.447
|
|
- type: map_at_1000
|
|
value: 41.454
|
|
- type: map_at_20
|
|
value: 41.197
|
|
- type: map_at_3
|
|
value: 35.099999999999994
|
|
- type: map_at_5
|
|
value: 38.196999999999996
|
|
- type: mrr_at_1
|
|
value: 24.537695590327168
|
|
- type: mrr_at_10
|
|
value: 40.67259929102034
|
|
- type: mrr_at_100
|
|
value: 41.639618460436125
|
|
- type: mrr_at_1000
|
|
value: 41.64596845247576
|
|
- type: mrr_at_20
|
|
value: 41.38915253517258
|
|
- type: mrr_at_3
|
|
value: 35.27738264580362
|
|
- type: mrr_at_5
|
|
value: 38.40327169274532
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 7.431509810863732
|
|
- type: nauc_map_at_1000_max
|
|
value: -2.981272220393634
|
|
- type: nauc_map_at_1000_std
|
|
value: -7.60710973485905
|
|
- type: nauc_map_at_100_diff1
|
|
value: 7.436737619273204
|
|
- type: nauc_map_at_100_max
|
|
value: -2.967184788185936
|
|
- type: nauc_map_at_100_std
|
|
value: -7.597426337410871
|
|
- type: nauc_map_at_10_diff1
|
|
value: 7.255093659685807
|
|
- type: nauc_map_at_10_max
|
|
value: -2.9042962900147544
|
|
- type: nauc_map_at_10_std
|
|
value: -7.934694729089717
|
|
- type: nauc_map_at_1_diff1
|
|
value: 12.509203312194646
|
|
- type: nauc_map_at_1_max
|
|
value: -5.881727148045224
|
|
- type: nauc_map_at_1_std
|
|
value: -7.791332615643759
|
|
- type: nauc_map_at_20_diff1
|
|
value: 7.327100008464186
|
|
- type: nauc_map_at_20_max
|
|
value: -2.837417061935196
|
|
- type: nauc_map_at_20_std
|
|
value: -7.727026459254324
|
|
- type: nauc_map_at_3_diff1
|
|
value: 6.852993254257847
|
|
- type: nauc_map_at_3_max
|
|
value: -4.051470844228069
|
|
- type: nauc_map_at_3_std
|
|
value: -7.896963683580916
|
|
- type: nauc_map_at_5_diff1
|
|
value: 6.528299731268904
|
|
- type: nauc_map_at_5_max
|
|
value: -3.6970340215693476
|
|
- type: nauc_map_at_5_std
|
|
value: -7.655276417735266
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 5.711449969160694
|
|
- type: nauc_mrr_at_1000_max
|
|
value: -3.4753506470039266
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -7.794020380041222
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 5.717019799542202
|
|
- type: nauc_mrr_at_100_max
|
|
value: -3.461221495753972
|
|
- type: nauc_mrr_at_100_std
|
|
value: -7.784340755281538
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 5.509993731954919
|
|
- type: nauc_mrr_at_10_max
|
|
value: -3.4562614853854345
|
|
- type: nauc_mrr_at_10_std
|
|
value: -8.172557318463994
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 10.815838583441858
|
|
- type: nauc_mrr_at_1_max
|
|
value: -5.323382194534891
|
|
- type: nauc_mrr_at_1_std
|
|
value: -8.038288156705363
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 5.622966175346149
|
|
- type: nauc_mrr_at_20_max
|
|
value: -3.3271519171448602
|
|
- type: nauc_mrr_at_20_std
|
|
value: -7.911979321248223
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 5.1203118177676945
|
|
- type: nauc_mrr_at_3_max
|
|
value: -4.663436282182911
|
|
- type: nauc_mrr_at_3_std
|
|
value: -8.16687342201878
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 4.899936200607895
|
|
- type: nauc_mrr_at_5_max
|
|
value: -4.238888324916206
|
|
- type: nauc_mrr_at_5_std
|
|
value: -7.911378372003927
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 7.208621858675132
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: -1.9047927444267347
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -6.986137159109878
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 7.409545817332008
|
|
- type: nauc_ndcg_at_100_max
|
|
value: -1.4631671846013694
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -6.630280309037233
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 6.4667756391170395
|
|
- type: nauc_ndcg_at_10_max
|
|
value: -0.6950268010456382
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -8.022144927522392
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 12.509203312194646
|
|
- type: nauc_ndcg_at_1_max
|
|
value: -5.881727148045224
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -7.791332615643759
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 6.726279074146785
|
|
- type: nauc_ndcg_at_20_max
|
|
value: -0.3861052348420354
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -7.221277273790139
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 5.5538863803913365
|
|
- type: nauc_ndcg_at_3_max
|
|
value: -3.5651217527245946
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -7.826880086024049
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 4.878905871379252
|
|
- type: nauc_ndcg_at_5_max
|
|
value: -2.821048486985759
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -7.31598311150453
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 31.595672412803232
|
|
- type: nauc_precision_at_1000_max
|
|
value: 42.56487657246246
|
|
- type: nauc_precision_at_1000_std
|
|
value: 76.77064740096077
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 37.959767569852325
|
|
- type: nauc_precision_at_100_max
|
|
value: 61.03819238774345
|
|
- type: nauc_precision_at_100_std
|
|
value: 57.75475522584779
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 3.679895666980749
|
|
- type: nauc_precision_at_10_max
|
|
value: 11.38829056417457
|
|
- type: nauc_precision_at_10_std
|
|
value: -8.650914185729293
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 12.509203312194646
|
|
- type: nauc_precision_at_1_max
|
|
value: -5.881727148045224
|
|
- type: nauc_precision_at_1_std
|
|
value: -7.791332615643759
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 4.065515107956777
|
|
- type: nauc_precision_at_20_max
|
|
value: 23.888067135216097
|
|
- type: nauc_precision_at_20_std
|
|
value: -1.4622436922054596
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 2.1003082872796663
|
|
- type: nauc_precision_at_3_max
|
|
value: -2.24675019839533
|
|
- type: nauc_precision_at_3_std
|
|
value: -7.604178336955303
|
|
- type: nauc_precision_at_5_diff1
|
|
value: -0.246824792648523
|
|
- type: nauc_precision_at_5_max
|
|
value: 0.0642032358424201
|
|
- type: nauc_precision_at_5_std
|
|
value: -6.0892549043276745
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 31.59567241280578
|
|
- type: nauc_recall_at_1000_max
|
|
value: 42.564876572459895
|
|
- type: nauc_recall_at_1000_std
|
|
value: 76.7706474009625
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 37.95976756985206
|
|
- type: nauc_recall_at_100_max
|
|
value: 61.03819238774383
|
|
- type: nauc_recall_at_100_std
|
|
value: 57.75475522584684
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 3.679895666980674
|
|
- type: nauc_recall_at_10_max
|
|
value: 11.388290564174538
|
|
- type: nauc_recall_at_10_std
|
|
value: -8.650914185729265
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 12.509203312194646
|
|
- type: nauc_recall_at_1_max
|
|
value: -5.881727148045224
|
|
- type: nauc_recall_at_1_std
|
|
value: -7.791332615643759
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 4.065515107957231
|
|
- type: nauc_recall_at_20_max
|
|
value: 23.888067135216005
|
|
- type: nauc_recall_at_20_std
|
|
value: -1.462243692205622
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 2.100308287279676
|
|
- type: nauc_recall_at_3_max
|
|
value: -2.2467501983953024
|
|
- type: nauc_recall_at_3_std
|
|
value: -7.604178336955286
|
|
- type: nauc_recall_at_5_diff1
|
|
value: -0.24682479264852286
|
|
- type: nauc_recall_at_5_max
|
|
value: 0.06420323584243659
|
|
- type: nauc_recall_at_5_std
|
|
value: -6.089254904327643
|
|
- type: ndcg_at_1
|
|
value: 24.04
|
|
- type: ndcg_at_10
|
|
value: 49.913000000000004
|
|
- type: ndcg_at_100
|
|
value: 54.057
|
|
- type: ndcg_at_1000
|
|
value: 54.213
|
|
- type: ndcg_at_20
|
|
value: 52.42400000000001
|
|
- type: ndcg_at_3
|
|
value: 38.842999999999996
|
|
- type: ndcg_at_5
|
|
value: 44.416
|
|
- type: precision_at_1
|
|
value: 24.04
|
|
- type: precision_at_10
|
|
value: 8.009
|
|
- type: precision_at_100
|
|
value: 0.984
|
|
- type: precision_at_1000
|
|
value: 0.1
|
|
- type: precision_at_20
|
|
value: 4.495
|
|
- type: precision_at_3
|
|
value: 16.572
|
|
- type: precision_at_5
|
|
value: 12.645999999999999
|
|
- type: recall_at_1
|
|
value: 24.04
|
|
- type: recall_at_10
|
|
value: 80.085
|
|
- type: recall_at_100
|
|
value: 98.36399999999999
|
|
- type: recall_at_1000
|
|
value: 99.57300000000001
|
|
- type: recall_at_20
|
|
value: 89.9
|
|
- type: recall_at_3
|
|
value: 49.716
|
|
- type: recall_at_5
|
|
value: 63.229
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CBD
|
|
revision: 36ddb419bcffe6a5374c3891957912892916f28d
|
|
split: test
|
|
type: PL-MTEB/cbd
|
|
metrics:
|
|
- type: accuracy
|
|
value: 64.91
|
|
- type: ap
|
|
value: 20.253009474993238
|
|
- type: ap_weighted
|
|
value: 20.253009474993238
|
|
- type: f1
|
|
value: 54.83698737303514
|
|
- type: f1_weighted
|
|
value: 69.53194816160229
|
|
- type: main_score
|
|
value: 64.91
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CDSC-E
|
|
revision: 0a3d4aa409b22f80eb22cbf59b492637637b536d
|
|
split: test
|
|
type: PL-MTEB/cdsce-pairclassification
|
|
metrics:
|
|
- type: cosine_accuracy
|
|
value: 87.5
|
|
- type: cosine_accuracy_threshold
|
|
value: 97.39001989364624
|
|
- type: cosine_ap
|
|
value: 71.63899566137869
|
|
- type: cosine_f1
|
|
value: 64.39024390243902
|
|
- type: cosine_f1_threshold
|
|
value: 94.18535828590393
|
|
- type: cosine_precision
|
|
value: 60.0
|
|
- type: cosine_recall
|
|
value: 69.47368421052632
|
|
- type: dot_accuracy
|
|
value: 87.5
|
|
- type: dot_accuracy_threshold
|
|
value: 97.39001989364624
|
|
- type: dot_ap
|
|
value: 71.63899566137869
|
|
- type: dot_f1
|
|
value: 64.39024390243902
|
|
- type: dot_f1_threshold
|
|
value: 94.18535232543945
|
|
- type: dot_precision
|
|
value: 60.0
|
|
- type: dot_recall
|
|
value: 69.47368421052632
|
|
- type: euclidean_accuracy
|
|
value: 87.5
|
|
- type: euclidean_accuracy_threshold
|
|
value: 22.847232222557068
|
|
- type: euclidean_ap
|
|
value: 71.63899566137869
|
|
- type: euclidean_f1
|
|
value: 64.39024390243902
|
|
- type: euclidean_f1_threshold
|
|
value: 34.101736545562744
|
|
- type: euclidean_precision
|
|
value: 60.0
|
|
- type: euclidean_recall
|
|
value: 69.47368421052632
|
|
- type: main_score
|
|
value: 71.83631821171632
|
|
- type: manhattan_accuracy
|
|
value: 87.6
|
|
- type: manhattan_accuracy_threshold
|
|
value: 499.97105598449707
|
|
- type: manhattan_ap
|
|
value: 71.83631821171632
|
|
- type: manhattan_f1
|
|
value: 64.5631067961165
|
|
- type: manhattan_f1_threshold
|
|
value: 809.0234756469727
|
|
- type: manhattan_precision
|
|
value: 59.909909909909906
|
|
- type: manhattan_recall
|
|
value: 70.0
|
|
- type: max_ap
|
|
value: 71.83631821171632
|
|
- type: max_f1
|
|
value: 64.5631067961165
|
|
- type: max_precision
|
|
value: 60.0
|
|
- type: max_recall
|
|
value: 70.0
|
|
- type: similarity_accuracy
|
|
value: 87.5
|
|
- type: similarity_accuracy_threshold
|
|
value: 97.39001989364624
|
|
- type: similarity_ap
|
|
value: 71.63899566137869
|
|
- type: similarity_f1
|
|
value: 64.39024390243902
|
|
- type: similarity_f1_threshold
|
|
value: 94.18535828590393
|
|
- type: similarity_precision
|
|
value: 60.0
|
|
- type: similarity_recall
|
|
value: 69.47368421052632
|
|
task:
|
|
type: PairClassification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CDSC-R
|
|
revision: 1cd6abbb00df7d14be3dbd76a7dcc64b3a79a7cd
|
|
split: test
|
|
type: PL-MTEB/cdscr-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 89.9839992597087
|
|
- type: cosine_spearman
|
|
value: 90.27044716786627
|
|
- type: euclidean_pearson
|
|
value: 87.74719535276023
|
|
- type: euclidean_spearman
|
|
value: 90.2703874383013
|
|
- type: main_score
|
|
value: 90.27044716786627
|
|
- type: manhattan_pearson
|
|
value: 87.81149530960033
|
|
- type: manhattan_spearman
|
|
value: 90.37098083828207
|
|
- type: pearson
|
|
value: 89.9839992597087
|
|
- type: spearman
|
|
value: 90.27044716786627
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB DBPedia-PL
|
|
revision: 76afe41d9af165cc40999fcaa92312b8b012064a
|
|
split: test
|
|
type: clarin-knext/dbpedia-pl
|
|
metrics:
|
|
- type: main_score
|
|
value: 29.225
|
|
- type: map_at_1
|
|
value: 5.92
|
|
- type: map_at_10
|
|
value: 13.052
|
|
- type: map_at_100
|
|
value: 18.054000000000002
|
|
- type: map_at_1000
|
|
value: 19.378999999999998
|
|
- type: map_at_20
|
|
value: 14.921000000000001
|
|
- type: map_at_3
|
|
value: 9.517000000000001
|
|
- type: map_at_5
|
|
value: 11.122
|
|
- type: mrr_at_1
|
|
value: 45.0
|
|
- type: mrr_at_10
|
|
value: 57.3967261904762
|
|
- type: mrr_at_100
|
|
value: 57.83804567388388
|
|
- type: mrr_at_1000
|
|
value: 57.86075000832548
|
|
- type: mrr_at_20
|
|
value: 57.66969785675282
|
|
- type: mrr_at_3
|
|
value: 55.16666666666667
|
|
- type: mrr_at_5
|
|
value: 56.64166666666669
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 29.411798531506246
|
|
- type: nauc_map_at_1000_max
|
|
value: 20.900134633305655
|
|
- type: nauc_map_at_1000_std
|
|
value: 31.404039472246353
|
|
- type: nauc_map_at_100_diff1
|
|
value: 30.843903551109808
|
|
- type: nauc_map_at_100_max
|
|
value: 17.39151067247246
|
|
- type: nauc_map_at_100_std
|
|
value: 27.44650726590824
|
|
- type: nauc_map_at_10_diff1
|
|
value: 37.979613569219495
|
|
- type: nauc_map_at_10_max
|
|
value: 9.222700346624988
|
|
- type: nauc_map_at_10_std
|
|
value: 12.007799385555293
|
|
- type: nauc_map_at_1_diff1
|
|
value: 53.50284116730185
|
|
- type: nauc_map_at_1_max
|
|
value: 1.370522275254312
|
|
- type: nauc_map_at_1_std
|
|
value: -0.30640006292692257
|
|
- type: nauc_map_at_20_diff1
|
|
value: 35.67559578714465
|
|
- type: nauc_map_at_20_max
|
|
value: 12.765002402346221
|
|
- type: nauc_map_at_20_std
|
|
value: 17.73265858605054
|
|
- type: nauc_map_at_3_diff1
|
|
value: 45.619789003130585
|
|
- type: nauc_map_at_3_max
|
|
value: 1.045838638341231
|
|
- type: nauc_map_at_3_std
|
|
value: 2.319308580529236
|
|
- type: nauc_map_at_5_diff1
|
|
value: 42.08058689946505
|
|
- type: nauc_map_at_5_max
|
|
value: 5.337616164644746
|
|
- type: nauc_map_at_5_std
|
|
value: 4.73118790791731
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 34.33930133013396
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 29.38799773918778
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 32.26009048699902
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 34.3197444457885
|
|
- type: nauc_mrr_at_100_max
|
|
value: 29.413059576309497
|
|
- type: nauc_mrr_at_100_std
|
|
value: 32.26908951100588
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 34.30610810384026
|
|
- type: nauc_mrr_at_10_max
|
|
value: 29.25358347303212
|
|
- type: nauc_mrr_at_10_std
|
|
value: 32.42735770220712
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 38.47836050546717
|
|
- type: nauc_mrr_at_1_max
|
|
value: 25.549990178746796
|
|
- type: nauc_mrr_at_1_std
|
|
value: 27.017285405617763
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 34.32685063678914
|
|
- type: nauc_mrr_at_20_max
|
|
value: 29.382152716878547
|
|
- type: nauc_mrr_at_20_std
|
|
value: 32.36225065070027
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 34.94513788944085
|
|
- type: nauc_mrr_at_3_max
|
|
value: 28.948106098297938
|
|
- type: nauc_mrr_at_3_std
|
|
value: 31.752978523564845
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 34.22773791436512
|
|
- type: nauc_mrr_at_5_max
|
|
value: 28.645995406061914
|
|
- type: nauc_mrr_at_5_std
|
|
value: 31.947761641656065
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 23.59930215160307
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 30.004827423326873
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 45.14606063029462
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 27.150265390833766
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 21.542350038665962
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 37.04783459199997
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 30.44928623138369
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 21.38523283782705
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 31.948655996496527
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 38.141954118151105
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 20.764788523221725
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 24.457971796268065
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 31.668458090974728
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 20.1903988669924
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 30.646872442412544
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 30.030850630038053
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 19.919461574491066
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 28.065728170179188
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 30.06324115773368
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 21.013491210996943
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 29.390767365137947
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -15.2968288893292
|
|
- type: nauc_precision_at_1000_max
|
|
value: 48.371418703337305
|
|
- type: nauc_precision_at_1000_std
|
|
value: 33.90852748893144
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -7.607176962046647
|
|
- type: nauc_precision_at_100_max
|
|
value: 35.35122884806948
|
|
- type: nauc_precision_at_100_std
|
|
value: 46.4742326977524
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 0.0234083902358811
|
|
- type: nauc_precision_at_10_max
|
|
value: 34.310462135642645
|
|
- type: nauc_precision_at_10_std
|
|
value: 46.22745495492598
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 38.47836050546717
|
|
- type: nauc_precision_at_1_max
|
|
value: 25.549990178746796
|
|
- type: nauc_precision_at_1_std
|
|
value: 27.017285405617763
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -0.7281234339501458
|
|
- type: nauc_precision_at_20_max
|
|
value: 34.879992298927796
|
|
- type: nauc_precision_at_20_std
|
|
value: 46.6455237720046
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 12.557632325001943
|
|
- type: nauc_precision_at_3_max
|
|
value: 27.472641291674343
|
|
- type: nauc_precision_at_3_std
|
|
value: 32.76253410590738
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 5.72403051661784
|
|
- type: nauc_precision_at_5_max
|
|
value: 31.623557984213747
|
|
- type: nauc_precision_at_5_std
|
|
value: 37.60956680129879
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 5.745409852861974
|
|
- type: nauc_recall_at_1000_max
|
|
value: 27.497512598172698
|
|
- type: nauc_recall_at_1000_std
|
|
value: 48.07303762126119
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 17.211282922855617
|
|
- type: nauc_recall_at_100_max
|
|
value: 17.98582110327383
|
|
- type: nauc_recall_at_100_std
|
|
value: 34.86455715009784
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 28.755279638184874
|
|
- type: nauc_recall_at_10_max
|
|
value: 8.106029595934537
|
|
- type: nauc_recall_at_10_std
|
|
value: 12.493783688335569
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 53.50284116730185
|
|
- type: nauc_recall_at_1_max
|
|
value: 1.370522275254312
|
|
- type: nauc_recall_at_1_std
|
|
value: -0.30640006292692257
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 27.994527440411993
|
|
- type: nauc_recall_at_20_max
|
|
value: 12.916323071056604
|
|
- type: nauc_recall_at_20_std
|
|
value: 17.70928825635808
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 39.80550258552395
|
|
- type: nauc_recall_at_3_max
|
|
value: -0.8593780074939045
|
|
- type: nauc_recall_at_3_std
|
|
value: 2.086691158003704
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 34.29080510342918
|
|
- type: nauc_recall_at_5_max
|
|
value: 2.8885937240283113
|
|
- type: nauc_recall_at_5_std
|
|
value: 2.6609799835271852
|
|
- type: ndcg_at_1
|
|
value: 35.875
|
|
- type: ndcg_at_10
|
|
value: 29.225
|
|
- type: ndcg_at_100
|
|
value: 33.554
|
|
- type: ndcg_at_1000
|
|
value: 40.908
|
|
- type: ndcg_at_20
|
|
value: 28.910000000000004
|
|
- type: ndcg_at_3
|
|
value: 32.405
|
|
- type: ndcg_at_5
|
|
value: 30.408
|
|
- type: precision_at_1
|
|
value: 45.0
|
|
- type: precision_at_10
|
|
value: 23.599999999999998
|
|
- type: precision_at_100
|
|
value: 7.68
|
|
- type: precision_at_1000
|
|
value: 1.804
|
|
- type: precision_at_20
|
|
value: 17.5
|
|
- type: precision_at_3
|
|
value: 36.167
|
|
- type: precision_at_5
|
|
value: 30.15
|
|
- type: recall_at_1
|
|
value: 5.92
|
|
- type: recall_at_10
|
|
value: 18.658
|
|
- type: recall_at_100
|
|
value: 40.144999999999996
|
|
- type: recall_at_1000
|
|
value: 63.914
|
|
- type: recall_at_20
|
|
value: 23.91
|
|
- type: recall_at_3
|
|
value: 11.334
|
|
- type: recall_at_5
|
|
value: 14.251
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB 8TagsClustering
|
|
revision: 78b962b130c6690659c65abf67bf1c2f030606b6
|
|
split: test
|
|
type: PL-MTEB/8tags-clustering
|
|
metrics:
|
|
- type: main_score
|
|
value: 37.57372573379629
|
|
- type: v_measure
|
|
value: 37.57372573379629
|
|
- type: v_measure_std
|
|
value: 1.576502898019969
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB FiQA-PL
|
|
revision: 2e535829717f8bf9dc829b7f911cc5bbd4e6608e
|
|
split: test
|
|
type: clarin-knext/fiqa-pl
|
|
metrics:
|
|
- type: main_score
|
|
value: 25.322
|
|
- type: map_at_1
|
|
value: 12.084
|
|
- type: map_at_10
|
|
value: 19.402
|
|
- type: map_at_100
|
|
value: 20.766000000000002
|
|
- type: map_at_1000
|
|
value: 20.958
|
|
- type: map_at_20
|
|
value: 20.085
|
|
- type: map_at_3
|
|
value: 16.794
|
|
- type: map_at_5
|
|
value: 18.242
|
|
- type: mrr_at_1
|
|
value: 23.30246913580247
|
|
- type: mrr_at_10
|
|
value: 31.084594846168915
|
|
- type: mrr_at_100
|
|
value: 32.081458268143486
|
|
- type: mrr_at_1000
|
|
value: 32.15082259510916
|
|
- type: mrr_at_20
|
|
value: 31.641799089124518
|
|
- type: mrr_at_3
|
|
value: 28.703703703703713
|
|
- type: mrr_at_5
|
|
value: 30.12345679012346
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 33.497391865616606
|
|
- type: nauc_map_at_1000_max
|
|
value: 15.431683878656488
|
|
- type: nauc_map_at_1000_std
|
|
value: 10.827813213986468
|
|
- type: nauc_map_at_100_diff1
|
|
value: 33.534068616502886
|
|
- type: nauc_map_at_100_max
|
|
value: 15.291439989133599
|
|
- type: nauc_map_at_100_std
|
|
value: 10.715061061847777
|
|
- type: nauc_map_at_10_diff1
|
|
value: 33.49437614167937
|
|
- type: nauc_map_at_10_max
|
|
value: 14.377484560226964
|
|
- type: nauc_map_at_10_std
|
|
value: 9.487834206589557
|
|
- type: nauc_map_at_1_diff1
|
|
value: 39.87810373637443
|
|
- type: nauc_map_at_1_max
|
|
value: 10.730137705508765
|
|
- type: nauc_map_at_1_std
|
|
value: 3.2660873686456195
|
|
- type: nauc_map_at_20_diff1
|
|
value: 33.37736866727796
|
|
- type: nauc_map_at_20_max
|
|
value: 14.70143784805556
|
|
- type: nauc_map_at_20_std
|
|
value: 9.989663285421791
|
|
- type: nauc_map_at_3_diff1
|
|
value: 34.368864609204216
|
|
- type: nauc_map_at_3_max
|
|
value: 12.768667645519768
|
|
- type: nauc_map_at_3_std
|
|
value: 7.982752811874638
|
|
- type: nauc_map_at_5_diff1
|
|
value: 33.58267051366728
|
|
- type: nauc_map_at_5_max
|
|
value: 13.529005222918848
|
|
- type: nauc_map_at_5_std
|
|
value: 8.565140707894367
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 34.518749214862446
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 20.004412541379317
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 10.794450592562008
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 34.502828469831684
|
|
- type: nauc_mrr_at_100_max
|
|
value: 20.016402128122674
|
|
- type: nauc_mrr_at_100_std
|
|
value: 10.770953740589398
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 34.464123530074744
|
|
- type: nauc_mrr_at_10_max
|
|
value: 19.812317084561315
|
|
- type: nauc_mrr_at_10_std
|
|
value: 10.660604975440622
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 39.735267543303344
|
|
- type: nauc_mrr_at_1_max
|
|
value: 20.218792748481526
|
|
- type: nauc_mrr_at_1_std
|
|
value: 7.574870456628672
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 34.4112636812203
|
|
- type: nauc_mrr_at_20_max
|
|
value: 19.736403323847995
|
|
- type: nauc_mrr_at_20_std
|
|
value: 10.58825811173397
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 34.322321922524765
|
|
- type: nauc_mrr_at_3_max
|
|
value: 19.48120229919887
|
|
- type: nauc_mrr_at_3_std
|
|
value: 10.241852033769396
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 34.41273362560696
|
|
- type: nauc_mrr_at_5_max
|
|
value: 19.80166599189298
|
|
- type: nauc_mrr_at_5_std
|
|
value: 10.535257678547225
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 31.756209625205372
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 19.79815198505404
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 15.747292429924494
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 32.24612802150064
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 18.490724459073633
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 14.606523975785374
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 32.17599943968043
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 15.73203247263979
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 11.361059016427816
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 39.735267543303344
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 20.218792748481526
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 7.574870456628672
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 31.750276068192886
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 15.761403266813346
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 11.939341736048261
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 32.60001850916417
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 16.484580482661286
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 9.93945065513519
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 32.44524427279313
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 15.875506598237141
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 9.982281820511833
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 5.371199115978502
|
|
- type: nauc_precision_at_1000_max
|
|
value: 32.2390464051828
|
|
- type: nauc_precision_at_1000_std
|
|
value: 14.878904307648414
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 16.16681952079101
|
|
- type: nauc_precision_at_100_max
|
|
value: 31.799356005933838
|
|
- type: nauc_precision_at_100_std
|
|
value: 19.248994737500986
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 22.009585966198923
|
|
- type: nauc_precision_at_10_max
|
|
value: 25.75349877480564
|
|
- type: nauc_precision_at_10_std
|
|
value: 16.27236030310856
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 39.735267543303344
|
|
- type: nauc_precision_at_1_max
|
|
value: 20.218792748481526
|
|
- type: nauc_precision_at_1_std
|
|
value: 7.574870456628672
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 18.58140182399686
|
|
- type: nauc_precision_at_20_max
|
|
value: 25.678514022441874
|
|
- type: nauc_precision_at_20_std
|
|
value: 16.797936080303757
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 26.928025721272824
|
|
- type: nauc_precision_at_3_max
|
|
value: 20.657641661666794
|
|
- type: nauc_precision_at_3_std
|
|
value: 13.0985390930848
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 23.36859898010871
|
|
- type: nauc_precision_at_5_max
|
|
value: 22.374908445175237
|
|
- type: nauc_precision_at_5_std
|
|
value: 14.246505892972294
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 11.980972712740272
|
|
- type: nauc_recall_at_1000_max
|
|
value: 19.76758314007667
|
|
- type: nauc_recall_at_1000_std
|
|
value: 37.01896226544845
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 21.23333081030157
|
|
- type: nauc_recall_at_100_max
|
|
value: 17.273702477754753
|
|
- type: nauc_recall_at_100_std
|
|
value: 22.66184024937999
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 24.654784002876422
|
|
- type: nauc_recall_at_10_max
|
|
value: 11.299238954418193
|
|
- type: nauc_recall_at_10_std
|
|
value: 12.933536657323804
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 39.87810373637443
|
|
- type: nauc_recall_at_1_max
|
|
value: 10.730137705508765
|
|
- type: nauc_recall_at_1_std
|
|
value: 3.2660873686456195
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 22.912968265183142
|
|
- type: nauc_recall_at_20_max
|
|
value: 10.463163094071744
|
|
- type: nauc_recall_at_20_std
|
|
value: 13.342666469120315
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 26.200195626449702
|
|
- type: nauc_recall_at_3_max
|
|
value: 10.661728055293116
|
|
- type: nauc_recall_at_3_std
|
|
value: 10.101882781882052
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 25.286289446845807
|
|
- type: nauc_recall_at_5_max
|
|
value: 11.353540373539142
|
|
- type: nauc_recall_at_5_std
|
|
value: 10.67026258089847
|
|
- type: ndcg_at_1
|
|
value: 23.302
|
|
- type: ndcg_at_10
|
|
value: 25.322
|
|
- type: ndcg_at_100
|
|
value: 31.452
|
|
- type: ndcg_at_1000
|
|
value: 35.378
|
|
- type: ndcg_at_20
|
|
value: 27.392
|
|
- type: ndcg_at_3
|
|
value: 22.238
|
|
- type: ndcg_at_5
|
|
value: 23.436
|
|
- type: precision_at_1
|
|
value: 23.302
|
|
- type: precision_at_10
|
|
value: 7.037
|
|
- type: precision_at_100
|
|
value: 1.321
|
|
- type: precision_at_1000
|
|
value: 0.2
|
|
- type: precision_at_20
|
|
value: 4.344
|
|
- type: precision_at_3
|
|
value: 14.557999999999998
|
|
- type: precision_at_5
|
|
value: 10.988000000000001
|
|
- type: recall_at_1
|
|
value: 12.084
|
|
- type: recall_at_10
|
|
value: 31.011
|
|
- type: recall_at_100
|
|
value: 54.782
|
|
- type: recall_at_1000
|
|
value: 78.828
|
|
- type: recall_at_20
|
|
value: 37.573
|
|
- type: recall_at_3
|
|
value: 20.918999999999997
|
|
- type: recall_at_5
|
|
value: 25.434
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB HotpotQA-PL
|
|
revision: a0bd479ac97b4ccb5bd6ce320c415d0bb4beb907
|
|
split: test
|
|
type: clarin-knext/hotpotqa-pl
|
|
metrics:
|
|
- type: main_score
|
|
value: 61.76199999999999
|
|
- type: map_at_1
|
|
value: 36.462
|
|
- type: map_at_10
|
|
value: 52.595000000000006
|
|
- type: map_at_100
|
|
value: 53.486
|
|
- type: map_at_1000
|
|
value: 53.561
|
|
- type: map_at_20
|
|
value: 53.116
|
|
- type: map_at_3
|
|
value: 49.55
|
|
- type: map_at_5
|
|
value: 51.468
|
|
- type: mrr_at_1
|
|
value: 72.92370020256584
|
|
- type: mrr_at_10
|
|
value: 79.14170498269061
|
|
- type: mrr_at_100
|
|
value: 79.39082829565201
|
|
- type: mrr_at_1000
|
|
value: 79.4039312237504
|
|
- type: mrr_at_20
|
|
value: 79.30320990617905
|
|
- type: mrr_at_3
|
|
value: 78.06887238352448
|
|
- type: mrr_at_5
|
|
value: 78.74746792707597
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 26.629197478945656
|
|
- type: nauc_map_at_1000_max
|
|
value: 20.417296536263652
|
|
- type: nauc_map_at_1000_std
|
|
value: 7.824861166949661
|
|
- type: nauc_map_at_100_diff1
|
|
value: 26.597747680876964
|
|
- type: nauc_map_at_100_max
|
|
value: 20.394321293004854
|
|
- type: nauc_map_at_100_std
|
|
value: 7.812277969136019
|
|
- type: nauc_map_at_10_diff1
|
|
value: 26.733323682484784
|
|
- type: nauc_map_at_10_max
|
|
value: 20.271344228458663
|
|
- type: nauc_map_at_10_std
|
|
value: 7.0935616016511815
|
|
- type: nauc_map_at_1_diff1
|
|
value: 73.40480136620272
|
|
- type: nauc_map_at_1_max
|
|
value: 38.86815860879837
|
|
- type: nauc_map_at_1_std
|
|
value: 4.8325955891477275
|
|
- type: nauc_map_at_20_diff1
|
|
value: 26.568842010897114
|
|
- type: nauc_map_at_20_max
|
|
value: 20.275169904863905
|
|
- type: nauc_map_at_20_std
|
|
value: 7.56661656432979
|
|
- type: nauc_map_at_3_diff1
|
|
value: 28.824845889064793
|
|
- type: nauc_map_at_3_max
|
|
value: 20.76852907202902
|
|
- type: nauc_map_at_3_std
|
|
value: 5.754512537392399
|
|
- type: nauc_map_at_5_diff1
|
|
value: 27.454615905979974
|
|
- type: nauc_map_at_5_max
|
|
value: 20.352277144385937
|
|
- type: nauc_map_at_5_std
|
|
value: 6.601409288581079
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 72.29337975556386
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 41.162812968303555
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 7.658983139015768
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 72.28963649528013
|
|
- type: nauc_mrr_at_100_max
|
|
value: 41.16405855619647
|
|
- type: nauc_mrr_at_100_std
|
|
value: 7.671105812656405
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 72.20735283859506
|
|
- type: nauc_mrr_at_10_max
|
|
value: 41.22707207638071
|
|
- type: nauc_mrr_at_10_std
|
|
value: 7.642216005282447
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 73.40480136620272
|
|
- type: nauc_mrr_at_1_max
|
|
value: 38.86815860879837
|
|
- type: nauc_mrr_at_1_std
|
|
value: 4.8325955891477275
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 72.28084176981353
|
|
- type: nauc_mrr_at_20_max
|
|
value: 41.19699794135133
|
|
- type: nauc_mrr_at_20_std
|
|
value: 7.673602725654943
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 72.2517312298736
|
|
- type: nauc_mrr_at_3_max
|
|
value: 41.23050336709122
|
|
- type: nauc_mrr_at_3_std
|
|
value: 7.055398076214827
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 72.3010580466702
|
|
- type: nauc_mrr_at_5_max
|
|
value: 41.16023128418148
|
|
- type: nauc_mrr_at_5_std
|
|
value: 7.224799100313062
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 31.836096618552684
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 24.19594101782851
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 11.27051039772318
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 31.010910429281985
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 23.73763527936943
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 11.202567249866915
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 31.630736903110733
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 23.29057670190408
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 8.622063436605352
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 73.40480136620272
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 38.86815860879837
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 4.8325955891477275
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 31.022867077795073
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 23.20240329652894
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 9.910412291823127
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 35.496569057786346
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 24.448277354535833
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 6.498237519761217
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 33.251227793460906
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 23.605853646520984
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 7.54284385208763
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -0.47079501803456375
|
|
- type: nauc_precision_at_1000_max
|
|
value: 15.089814566667142
|
|
- type: nauc_precision_at_1000_std
|
|
value: 27.847788246114057
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 3.0595485970514704
|
|
- type: nauc_precision_at_100_max
|
|
value: 14.360431203666717
|
|
- type: nauc_precision_at_100_std
|
|
value: 22.31753410548815
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 11.454235819834814
|
|
- type: nauc_precision_at_10_max
|
|
value: 14.979788854311145
|
|
- type: nauc_precision_at_10_std
|
|
value: 11.290542607411098
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 73.40480136620272
|
|
- type: nauc_precision_at_1_max
|
|
value: 38.86815860879837
|
|
- type: nauc_precision_at_1_std
|
|
value: 4.8325955891477275
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 7.60972218209098
|
|
- type: nauc_precision_at_20_max
|
|
value: 13.692113405742418
|
|
- type: nauc_precision_at_20_std
|
|
value: 15.359273788872974
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 22.002230799209492
|
|
- type: nauc_precision_at_3_max
|
|
value: 19.075064977055266
|
|
- type: nauc_precision_at_3_std
|
|
value: 7.1760372858256956
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 16.565606958337607
|
|
- type: nauc_precision_at_5_max
|
|
value: 16.550935196750206
|
|
- type: nauc_precision_at_5_std
|
|
value: 8.807234374696868
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: -0.47079501803429247
|
|
- type: nauc_recall_at_1000_max
|
|
value: 15.089814566667334
|
|
- type: nauc_recall_at_1000_std
|
|
value: 27.847788246114025
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 3.0595485970514558
|
|
- type: nauc_recall_at_100_max
|
|
value: 14.360431203666705
|
|
- type: nauc_recall_at_100_std
|
|
value: 22.317534105488054
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 11.4542358198349
|
|
- type: nauc_recall_at_10_max
|
|
value: 14.979788854311154
|
|
- type: nauc_recall_at_10_std
|
|
value: 11.290542607411085
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 73.40480136620272
|
|
- type: nauc_recall_at_1_max
|
|
value: 38.86815860879837
|
|
- type: nauc_recall_at_1_std
|
|
value: 4.8325955891477275
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 7.609722182091017
|
|
- type: nauc_recall_at_20_max
|
|
value: 13.692113405742424
|
|
- type: nauc_recall_at_20_std
|
|
value: 15.35927378887301
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 22.002230799209435
|
|
- type: nauc_recall_at_3_max
|
|
value: 19.07506497705519
|
|
- type: nauc_recall_at_3_std
|
|
value: 7.176037285825619
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 16.56560695833764
|
|
- type: nauc_recall_at_5_max
|
|
value: 16.55093519675023
|
|
- type: nauc_recall_at_5_std
|
|
value: 8.807234374696902
|
|
- type: ndcg_at_1
|
|
value: 72.924
|
|
- type: ndcg_at_10
|
|
value: 61.76199999999999
|
|
- type: ndcg_at_100
|
|
value: 64.943
|
|
- type: ndcg_at_1000
|
|
value: 66.42
|
|
- type: ndcg_at_20
|
|
value: 63.105
|
|
- type: ndcg_at_3
|
|
value: 57.318000000000005
|
|
- type: ndcg_at_5
|
|
value: 59.80799999999999
|
|
- type: precision_at_1
|
|
value: 72.924
|
|
- type: precision_at_10
|
|
value: 12.723999999999998
|
|
- type: precision_at_100
|
|
value: 1.521
|
|
- type: precision_at_1000
|
|
value: 0.172
|
|
- type: precision_at_20
|
|
value: 6.795
|
|
- type: precision_at_3
|
|
value: 35.863
|
|
- type: precision_at_5
|
|
value: 23.487
|
|
- type: recall_at_1
|
|
value: 36.462
|
|
- type: recall_at_10
|
|
value: 63.619
|
|
- type: recall_at_100
|
|
value: 76.036
|
|
- type: recall_at_1000
|
|
value: 85.8
|
|
- type: recall_at_20
|
|
value: 67.95400000000001
|
|
- type: recall_at_3
|
|
value: 53.795
|
|
- type: recall_at_5
|
|
value: 58.717
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB MSMARCO-PL
|
|
revision: 8634c07806d5cce3a6138e260e59b81760a0a640
|
|
split: test
|
|
type: clarin-knext/msmarco-pl
|
|
metrics:
|
|
- type: main_score
|
|
value: 45.132
|
|
- type: map_at_1
|
|
value: 1.667
|
|
- type: map_at_10
|
|
value: 8.405999999999999
|
|
- type: map_at_100
|
|
value: 20.796
|
|
- type: map_at_1000
|
|
value: 25.679999999999996
|
|
- type: map_at_20
|
|
value: 11.882
|
|
- type: map_at_3
|
|
value: 3.4000000000000004
|
|
- type: map_at_5
|
|
value: 5.289
|
|
- type: mrr_at_1
|
|
value: 62.7906976744186
|
|
- type: mrr_at_10
|
|
value: 71.9767441860465
|
|
- type: mrr_at_100
|
|
value: 72.19001178866145
|
|
- type: mrr_at_1000
|
|
value: 72.21077590826278
|
|
- type: mrr_at_20
|
|
value: 71.9767441860465
|
|
- type: mrr_at_3
|
|
value: 69.76744186046511
|
|
- type: mrr_at_5
|
|
value: 71.9767441860465
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 13.121496890926018
|
|
- type: nauc_map_at_1000_max
|
|
value: 64.4620914971356
|
|
- type: nauc_map_at_1000_std
|
|
value: 70.89107882842627
|
|
- type: nauc_map_at_100_diff1
|
|
value: 6.569373263154751
|
|
- type: nauc_map_at_100_max
|
|
value: 54.52329917268778
|
|
- type: nauc_map_at_100_std
|
|
value: 57.970012281008195
|
|
- type: nauc_map_at_10_diff1
|
|
value: 12.479881525075633
|
|
- type: nauc_map_at_10_max
|
|
value: 16.416934605814358
|
|
- type: nauc_map_at_10_std
|
|
value: 16.562025084061755
|
|
- type: nauc_map_at_1_diff1
|
|
value: -13.480148625088354
|
|
- type: nauc_map_at_1_max
|
|
value: -12.48386553446901
|
|
- type: nauc_map_at_1_std
|
|
value: -19.47568765990734
|
|
- type: nauc_map_at_20_diff1
|
|
value: 8.75113737642458
|
|
- type: nauc_map_at_20_max
|
|
value: 28.316394733873455
|
|
- type: nauc_map_at_20_std
|
|
value: 28.706433416288757
|
|
- type: nauc_map_at_3_diff1
|
|
value: 0.4892858373106769
|
|
- type: nauc_map_at_3_max
|
|
value: 4.82429174133813
|
|
- type: nauc_map_at_3_std
|
|
value: 2.685691736161667
|
|
- type: nauc_map_at_5_diff1
|
|
value: 7.407280581282287
|
|
- type: nauc_map_at_5_max
|
|
value: 7.810182361989069
|
|
- type: nauc_map_at_5_std
|
|
value: 7.1694430987177915
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: -1.3143171207174462
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 55.56132775818817
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 44.747614607383106
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: -1.224506180649995
|
|
- type: nauc_mrr_at_100_max
|
|
value: 55.600720798015224
|
|
- type: nauc_mrr_at_100_std
|
|
value: 44.73970951740156
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: -1.404072265069855
|
|
- type: nauc_mrr_at_10_max
|
|
value: 55.81202913496246
|
|
- type: nauc_mrr_at_10_std
|
|
value: 45.1755213724528
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: -3.3932017924925764
|
|
- type: nauc_mrr_at_1_max
|
|
value: 45.85906083891651
|
|
- type: nauc_mrr_at_1_std
|
|
value: 36.94174294169342
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: -1.404072265069855
|
|
- type: nauc_mrr_at_20_max
|
|
value: 55.81202913496246
|
|
- type: nauc_mrr_at_20_std
|
|
value: 45.1755213724528
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: -1.9535315867645546
|
|
- type: nauc_mrr_at_3_max
|
|
value: 54.66533478368106
|
|
- type: nauc_mrr_at_3_std
|
|
value: 42.93031026511843
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: -1.404072265069855
|
|
- type: nauc_mrr_at_5_max
|
|
value: 55.81202913496246
|
|
- type: nauc_mrr_at_5_std
|
|
value: 45.1755213724528
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 15.612187648926648
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 66.0369696987196
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 69.96669745374349
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 8.757636842486582
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 60.74693277069104
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 63.76108092965522
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 6.45234697262411
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 47.130858592103536
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 46.654922458779126
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: -4.400276896768569
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 24.736725318748277
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 15.100951232927404
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: -0.44419635404462504
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 53.81470890104093
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 54.65514527813791
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 4.176276992379476
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 33.4079755228582
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 26.097236468435497
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 9.966039505450683
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 40.118178652342394
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 34.33405125137147
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 13.757669487153102
|
|
- type: nauc_precision_at_1000_max
|
|
value: 52.007228955531794
|
|
- type: nauc_precision_at_1000_std
|
|
value: 62.70603005119199
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 7.1595084301066105
|
|
- type: nauc_precision_at_100_max
|
|
value: 57.56055309573276
|
|
- type: nauc_precision_at_100_std
|
|
value: 69.09674838687823
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 10.548904389246808
|
|
- type: nauc_precision_at_10_max
|
|
value: 58.361747853932435
|
|
- type: nauc_precision_at_10_std
|
|
value: 62.35890309913381
|
|
- type: nauc_precision_at_1_diff1
|
|
value: -3.3932017924925764
|
|
- type: nauc_precision_at_1_max
|
|
value: 45.85906083891651
|
|
- type: nauc_precision_at_1_std
|
|
value: 36.94174294169342
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 0.5486557649755647
|
|
- type: nauc_precision_at_20_max
|
|
value: 55.8966200841496
|
|
- type: nauc_precision_at_20_std
|
|
value: 64.46833667077514
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 3.74969726265482
|
|
- type: nauc_precision_at_3_max
|
|
value: 50.98538299147468
|
|
- type: nauc_precision_at_3_std
|
|
value: 47.52256580019106
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 14.409304075805396
|
|
- type: nauc_precision_at_5_max
|
|
value: 52.63426384539844
|
|
- type: nauc_precision_at_5_std
|
|
value: 48.72540538657435
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 14.810856570503505
|
|
- type: nauc_recall_at_1000_max
|
|
value: 56.70402594077228
|
|
- type: nauc_recall_at_1000_std
|
|
value: 62.44988045776601
|
|
- type: nauc_recall_at_100_diff1
|
|
value: -0.547033022823402
|
|
- type: nauc_recall_at_100_max
|
|
value: 37.5943435400723
|
|
- type: nauc_recall_at_100_std
|
|
value: 42.055737611040904
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 5.6072575274918695
|
|
- type: nauc_recall_at_10_max
|
|
value: 6.244507044627988
|
|
- type: nauc_recall_at_10_std
|
|
value: 5.1959433044082575
|
|
- type: nauc_recall_at_1_diff1
|
|
value: -13.480148625088354
|
|
- type: nauc_recall_at_1_max
|
|
value: -12.48386553446901
|
|
- type: nauc_recall_at_1_std
|
|
value: -19.47568765990734
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 1.5008424440815344
|
|
- type: nauc_recall_at_20_max
|
|
value: 16.711622731636748
|
|
- type: nauc_recall_at_20_std
|
|
value: 16.46978349884905
|
|
- type: nauc_recall_at_3_diff1
|
|
value: -2.3329900069251996
|
|
- type: nauc_recall_at_3_max
|
|
value: 2.511711071593615
|
|
- type: nauc_recall_at_3_std
|
|
value: -0.5855889251226093
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 4.1075104414046315
|
|
- type: nauc_recall_at_5_max
|
|
value: 0.34189966462509463
|
|
- type: nauc_recall_at_5_std
|
|
value: -1.89085195502975
|
|
- type: ndcg_at_1
|
|
value: 50.0
|
|
- type: ndcg_at_10
|
|
value: 45.132
|
|
- type: ndcg_at_100
|
|
value: 41.504999999999995
|
|
- type: ndcg_at_1000
|
|
value: 49.738
|
|
- type: ndcg_at_20
|
|
value: 42.569
|
|
- type: ndcg_at_3
|
|
value: 45.423
|
|
- type: ndcg_at_5
|
|
value: 45.611000000000004
|
|
- type: precision_at_1
|
|
value: 62.791
|
|
- type: precision_at_10
|
|
value: 54.419
|
|
- type: precision_at_100
|
|
value: 25.047000000000004
|
|
- type: precision_at_1000
|
|
value: 5.002
|
|
- type: precision_at_20
|
|
value: 46.394999999999996
|
|
- type: precision_at_3
|
|
value: 57.364000000000004
|
|
- type: precision_at_5
|
|
value: 57.208999999999996
|
|
- type: recall_at_1
|
|
value: 1.667
|
|
- type: recall_at_10
|
|
value: 10.933
|
|
- type: recall_at_100
|
|
value: 35.169
|
|
- type: recall_at_1000
|
|
value: 59.955999999999996
|
|
- type: recall_at_20
|
|
value: 16.399
|
|
- type: recall_at_3
|
|
value: 3.7379999999999995
|
|
- type: recall_at_5
|
|
value: 6.365
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: pl
|
|
name: MTEB MassiveIntentClassification (pl)
|
|
revision: 4672e20407010da34463acc759c162ca9734bca6
|
|
split: test
|
|
type: mteb/amazon_massive_intent
|
|
metrics:
|
|
- type: accuracy
|
|
value: 62.55548083389375
|
|
- type: f1
|
|
value: 55.243883281423955
|
|
- type: f1_weighted
|
|
value: 61.53554902108963
|
|
- type: main_score
|
|
value: 62.55548083389375
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: pl
|
|
name: MTEB MassiveScenarioClassification (pl)
|
|
revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
|
|
split: test
|
|
type: mteb/amazon_massive_scenario
|
|
metrics:
|
|
- type: accuracy
|
|
value: 71.7518493611298
|
|
- type: f1
|
|
value: 69.39084021404145
|
|
- type: f1_weighted
|
|
value: 71.48397679382578
|
|
- type: main_score
|
|
value: 71.7518493611298
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB NFCorpus-PL
|
|
revision: 9a6f9567fda928260afed2de480d79c98bf0bec0
|
|
split: test
|
|
type: clarin-knext/nfcorpus-pl
|
|
metrics:
|
|
- type: main_score
|
|
value: 27.359
|
|
- type: map_at_1
|
|
value: 4.013
|
|
- type: map_at_10
|
|
value: 9.243
|
|
- type: map_at_100
|
|
value: 11.417
|
|
- type: map_at_1000
|
|
value: 12.465
|
|
- type: map_at_20
|
|
value: 10.241999999999999
|
|
- type: map_at_3
|
|
value: 6.6739999999999995
|
|
- type: map_at_5
|
|
value: 7.720000000000001
|
|
- type: mrr_at_1
|
|
value: 36.84210526315789
|
|
- type: mrr_at_10
|
|
value: 45.80704211509165
|
|
- type: mrr_at_100
|
|
value: 46.43056530919217
|
|
- type: mrr_at_1000
|
|
value: 46.481813685972384
|
|
- type: mrr_at_20
|
|
value: 46.2328011230761
|
|
- type: mrr_at_3
|
|
value: 43.653250773993804
|
|
- type: mrr_at_5
|
|
value: 44.75232198142416
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 24.84177430292285
|
|
- type: nauc_map_at_1000_max
|
|
value: 17.115036682746375
|
|
- type: nauc_map_at_1000_std
|
|
value: 24.075727964418853
|
|
- type: nauc_map_at_100_diff1
|
|
value: 25.813465171019708
|
|
- type: nauc_map_at_100_max
|
|
value: 15.890774948775189
|
|
- type: nauc_map_at_100_std
|
|
value: 20.733065453457606
|
|
- type: nauc_map_at_10_diff1
|
|
value: 29.488943622716107
|
|
- type: nauc_map_at_10_max
|
|
value: 9.776720754233569
|
|
- type: nauc_map_at_10_std
|
|
value: 10.581345052422016
|
|
- type: nauc_map_at_1_diff1
|
|
value: 48.57974934948135
|
|
- type: nauc_map_at_1_max
|
|
value: 1.149527115405564
|
|
- type: nauc_map_at_1_std
|
|
value: -2.0301983395175363
|
|
- type: nauc_map_at_20_diff1
|
|
value: 27.744545091489826
|
|
- type: nauc_map_at_20_max
|
|
value: 12.800210322701194
|
|
- type: nauc_map_at_20_std
|
|
value: 15.036851255880851
|
|
- type: nauc_map_at_3_diff1
|
|
value: 37.25540055051418
|
|
- type: nauc_map_at_3_max
|
|
value: 4.906473702901897
|
|
- type: nauc_map_at_3_std
|
|
value: 1.462933406016024
|
|
- type: nauc_map_at_5_diff1
|
|
value: 33.75262117705747
|
|
- type: nauc_map_at_5_max
|
|
value: 5.349094540200769
|
|
- type: nauc_map_at_5_std
|
|
value: 4.009473353212513
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 25.923316236906224
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 30.218473131172814
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 34.32841034971355
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 25.89160877435761
|
|
- type: nauc_mrr_at_100_max
|
|
value: 30.26076316909358
|
|
- type: nauc_mrr_at_100_std
|
|
value: 34.38168790885202
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 25.94165965662626
|
|
- type: nauc_mrr_at_10_max
|
|
value: 29.92861838955619
|
|
- type: nauc_mrr_at_10_std
|
|
value: 34.217857324602384
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 27.77544038178182
|
|
- type: nauc_mrr_at_1_max
|
|
value: 23.544571519690063
|
|
- type: nauc_mrr_at_1_std
|
|
value: 29.133288904288985
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 25.817823276199377
|
|
- type: nauc_mrr_at_20_max
|
|
value: 30.212951519162534
|
|
- type: nauc_mrr_at_20_std
|
|
value: 34.38656845672502
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 27.253167791083772
|
|
- type: nauc_mrr_at_3_max
|
|
value: 28.668229911423044
|
|
- type: nauc_mrr_at_3_std
|
|
value: 32.24039598508148
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 26.50152942042588
|
|
- type: nauc_mrr_at_5_max
|
|
value: 29.014104429398657
|
|
- type: nauc_mrr_at_5_std
|
|
value: 33.10408829199384
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 21.670441606508682
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 35.085480170350294
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 40.26959838435534
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 20.56655267151386
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 29.059496472106172
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 36.20604882231693
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 19.327892822047392
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 22.970443207173847
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 33.63485024562264
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 29.440869586898806
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 21.1892146993199
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 27.715145294772626
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 19.84119342340242
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 24.648907071153918
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 34.21144991558109
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 22.475236266303952
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 22.5673625414089
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 30.40344427150939
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 20.435706146454795
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 20.807509478884405
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 30.50756403953348
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -7.734779276193169
|
|
- type: nauc_precision_at_1000_max
|
|
value: 10.369447288094234
|
|
- type: nauc_precision_at_1000_std
|
|
value: 38.88122374339474
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -5.148267935551239
|
|
- type: nauc_precision_at_100_max
|
|
value: 22.682811622480507
|
|
- type: nauc_precision_at_100_std
|
|
value: 52.14414978661011
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 4.2440553409575115
|
|
- type: nauc_precision_at_10_max
|
|
value: 24.922198902459577
|
|
- type: nauc_precision_at_10_std
|
|
value: 44.24729160099345
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 28.683873179972423
|
|
- type: nauc_precision_at_1_max
|
|
value: 24.333474443231477
|
|
- type: nauc_precision_at_1_std
|
|
value: 29.657103597064992
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 0.981459375147628
|
|
- type: nauc_precision_at_20_max
|
|
value: 26.656822900511944
|
|
- type: nauc_precision_at_20_std
|
|
value: 47.61829905274704
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 14.009226282963393
|
|
- type: nauc_precision_at_3_max
|
|
value: 25.206963221334643
|
|
- type: nauc_precision_at_3_std
|
|
value: 34.640163356829575
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 9.732199396026699
|
|
- type: nauc_precision_at_5_max
|
|
value: 21.620896160839308
|
|
- type: nauc_precision_at_5_std
|
|
value: 36.54829562203162
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 13.592706145413594
|
|
- type: nauc_recall_at_1000_max
|
|
value: 26.905710458923515
|
|
- type: nauc_recall_at_1000_std
|
|
value: 27.77232599212786
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 11.474980161550619
|
|
- type: nauc_recall_at_100_max
|
|
value: 24.6542606788053
|
|
- type: nauc_recall_at_100_std
|
|
value: 26.088933416325894
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 20.86627786542471
|
|
- type: nauc_recall_at_10_max
|
|
value: 12.310575849201342
|
|
- type: nauc_recall_at_10_std
|
|
value: 8.93720284107538
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 48.57974934948135
|
|
- type: nauc_recall_at_1_max
|
|
value: 1.149527115405564
|
|
- type: nauc_recall_at_1_std
|
|
value: -2.0301983395175363
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 17.03977114136929
|
|
- type: nauc_recall_at_20_max
|
|
value: 15.132361504438405
|
|
- type: nauc_recall_at_20_std
|
|
value: 14.39504435329145
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 33.90735954186142
|
|
- type: nauc_recall_at_3_max
|
|
value: 7.589690453066397
|
|
- type: nauc_recall_at_3_std
|
|
value: 0.8609172933612455
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 27.37452904528661
|
|
- type: nauc_recall_at_5_max
|
|
value: 6.950034812753282
|
|
- type: nauc_recall_at_5_std
|
|
value: 2.9248007586594396
|
|
- type: ndcg_at_1
|
|
value: 35.294
|
|
- type: ndcg_at_10
|
|
value: 27.359
|
|
- type: ndcg_at_100
|
|
value: 24.285999999999998
|
|
- type: ndcg_at_1000
|
|
value: 32.438
|
|
- type: ndcg_at_20
|
|
value: 25.418000000000003
|
|
- type: ndcg_at_3
|
|
value: 31.328
|
|
- type: ndcg_at_5
|
|
value: 29.269000000000002
|
|
- type: precision_at_1
|
|
value: 36.533
|
|
- type: precision_at_10
|
|
value: 20.681
|
|
- type: precision_at_100
|
|
value: 6.087
|
|
- type: precision_at_1000
|
|
value: 1.7469999999999999
|
|
- type: precision_at_20
|
|
value: 15.325
|
|
- type: precision_at_3
|
|
value: 29.309
|
|
- type: precision_at_5
|
|
value: 25.201
|
|
- type: recall_at_1
|
|
value: 4.013
|
|
- type: recall_at_10
|
|
value: 13.153
|
|
- type: recall_at_100
|
|
value: 24.549000000000003
|
|
- type: recall_at_1000
|
|
value: 53.908
|
|
- type: recall_at_20
|
|
value: 16.453
|
|
- type: recall_at_3
|
|
value: 7.832999999999999
|
|
- type: recall_at_5
|
|
value: 9.693999999999999
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB NQ-PL
|
|
revision: f171245712cf85dd4700b06bef18001578d0ca8d
|
|
split: test
|
|
type: clarin-knext/nq-pl
|
|
metrics:
|
|
- type: main_score
|
|
value: 30.842000000000002
|
|
- type: map_at_1
|
|
value: 15.584999999999999
|
|
- type: map_at_10
|
|
value: 25.141999999999996
|
|
- type: map_at_100
|
|
value: 26.387
|
|
- type: map_at_1000
|
|
value: 26.458
|
|
- type: map_at_20
|
|
value: 25.897
|
|
- type: map_at_3
|
|
value: 21.792
|
|
- type: map_at_5
|
|
value: 23.605
|
|
- type: mrr_at_1
|
|
value: 17.526071842410197
|
|
- type: mrr_at_10
|
|
value: 27.034281943754777
|
|
- type: mrr_at_100
|
|
value: 28.093499231975112
|
|
- type: mrr_at_1000
|
|
value: 28.151579697181628
|
|
- type: mrr_at_20
|
|
value: 27.685578601768064
|
|
- type: mrr_at_3
|
|
value: 23.966782541521876
|
|
- type: mrr_at_5
|
|
value: 25.63538045577454
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 25.629659206470034
|
|
- type: nauc_map_at_1000_max
|
|
value: 19.50903133109958
|
|
- type: nauc_map_at_1000_std
|
|
value: 11.369355803540456
|
|
- type: nauc_map_at_100_diff1
|
|
value: 25.63185640379452
|
|
- type: nauc_map_at_100_max
|
|
value: 19.49043016244933
|
|
- type: nauc_map_at_100_std
|
|
value: 11.349471698782217
|
|
- type: nauc_map_at_10_diff1
|
|
value: 25.801905100212085
|
|
- type: nauc_map_at_10_max
|
|
value: 18.71914313595772
|
|
- type: nauc_map_at_10_std
|
|
value: 10.101933080218412
|
|
- type: nauc_map_at_1_diff1
|
|
value: 27.69756013829008
|
|
- type: nauc_map_at_1_max
|
|
value: 13.265356278967614
|
|
- type: nauc_map_at_1_std
|
|
value: 4.845453511488002
|
|
- type: nauc_map_at_20_diff1
|
|
value: 25.57617091165384
|
|
- type: nauc_map_at_20_max
|
|
value: 19.22087134146287
|
|
- type: nauc_map_at_20_std
|
|
value: 10.863338999338074
|
|
- type: nauc_map_at_3_diff1
|
|
value: 26.04936647826419
|
|
- type: nauc_map_at_3_max
|
|
value: 17.00014461889098
|
|
- type: nauc_map_at_3_std
|
|
value: 8.345803797704802
|
|
- type: nauc_map_at_5_diff1
|
|
value: 25.926914766086163
|
|
- type: nauc_map_at_5_max
|
|
value: 17.909768342318312
|
|
- type: nauc_map_at_5_std
|
|
value: 8.99533665314055
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 24.821439280682775
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 20.48215524313607
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 13.302755245100787
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 24.822888515699727
|
|
- type: nauc_mrr_at_100_max
|
|
value: 20.476125364875305
|
|
- type: nauc_mrr_at_100_std
|
|
value: 13.303370196580808
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 24.827095834283377
|
|
- type: nauc_mrr_at_10_max
|
|
value: 19.906455259365014
|
|
- type: nauc_mrr_at_10_std
|
|
value: 12.461215626420783
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 27.354076617153282
|
|
- type: nauc_mrr_at_1_max
|
|
value: 15.421589080989397
|
|
- type: nauc_mrr_at_1_std
|
|
value: 7.854191402321044
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 24.707829956282353
|
|
- type: nauc_mrr_at_20_max
|
|
value: 20.343614549048684
|
|
- type: nauc_mrr_at_20_std
|
|
value: 12.991368337778994
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 25.001495195422212
|
|
- type: nauc_mrr_at_3_max
|
|
value: 18.670877184315987
|
|
- type: nauc_mrr_at_3_std
|
|
value: 11.073823459359353
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 25.09633485104506
|
|
- type: nauc_mrr_at_5_max
|
|
value: 19.289598809877393
|
|
- type: nauc_mrr_at_5_std
|
|
value: 11.447861090124427
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 24.454331896090252
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 24.54817880813177
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 18.291577235898664
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 24.4900499476292
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 24.3113863055596
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 18.283249505464127
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 24.75304628631047
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 21.346414904765112
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 13.144087870627114
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 27.354076617153282
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 15.421589080989397
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 7.854191402321044
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 24.054443970465634
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 23.02090178343728
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 15.466706732549639
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 25.21593203645425
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 18.366389791319857
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 9.886764558221312
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 25.18968308632415
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 19.714457143715883
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 10.810267333820615
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 5.311743560049695
|
|
- type: nauc_precision_at_1000_max
|
|
value: 31.8449636551786
|
|
- type: nauc_precision_at_1000_std
|
|
value: 38.560980646256645
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 11.642708984639716
|
|
- type: nauc_precision_at_100_max
|
|
value: 33.08348545702312
|
|
- type: nauc_precision_at_100_std
|
|
value: 38.84569611188958
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 19.39529546701617
|
|
- type: nauc_precision_at_10_max
|
|
value: 27.35329522618733
|
|
- type: nauc_precision_at_10_std
|
|
value: 21.657982938733863
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 27.354076617153282
|
|
- type: nauc_precision_at_1_max
|
|
value: 15.421589080989397
|
|
- type: nauc_precision_at_1_std
|
|
value: 7.854191402321044
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 15.315200424520157
|
|
- type: nauc_precision_at_20_max
|
|
value: 30.813032263448335
|
|
- type: nauc_precision_at_20_std
|
|
value: 28.51929835139947
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 23.171414749401624
|
|
- type: nauc_precision_at_3_max
|
|
value: 22.230781193639906
|
|
- type: nauc_precision_at_3_std
|
|
value: 14.39995607518812
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 22.12050049652593
|
|
- type: nauc_precision_at_5_max
|
|
value: 24.47739013891615
|
|
- type: nauc_precision_at_5_std
|
|
value: 15.911936861665232
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 18.49721947186244
|
|
- type: nauc_recall_at_1000_max
|
|
value: 59.77562391547361
|
|
- type: nauc_recall_at_1000_std
|
|
value: 67.25992226904116
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 21.08120571727416
|
|
- type: nauc_recall_at_100_max
|
|
value: 41.81711687017934
|
|
- type: nauc_recall_at_100_std
|
|
value: 45.46881224307712
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 22.267969061265276
|
|
- type: nauc_recall_at_10_max
|
|
value: 26.20350836241132
|
|
- type: nauc_recall_at_10_std
|
|
value: 18.312586912516927
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 27.69756013829008
|
|
- type: nauc_recall_at_1_max
|
|
value: 13.265356278967614
|
|
- type: nauc_recall_at_1_std
|
|
value: 4.845453511488002
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 19.7184358966775
|
|
- type: nauc_recall_at_20_max
|
|
value: 32.18279692099271
|
|
- type: nauc_recall_at_20_std
|
|
value: 26.185137240814377
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 23.501740451271914
|
|
- type: nauc_recall_at_3_max
|
|
value: 19.91360673787573
|
|
- type: nauc_recall_at_3_std
|
|
value: 11.210024942573977
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 23.437183434421655
|
|
- type: nauc_recall_at_5_max
|
|
value: 22.272023416475623
|
|
- type: nauc_recall_at_5_std
|
|
value: 12.814496156956142
|
|
- type: ndcg_at_1
|
|
value: 17.526
|
|
- type: ndcg_at_10
|
|
value: 30.842000000000002
|
|
- type: ndcg_at_100
|
|
value: 36.629
|
|
- type: ndcg_at_1000
|
|
value: 38.495000000000005
|
|
- type: ndcg_at_20
|
|
value: 33.382
|
|
- type: ndcg_at_3
|
|
value: 24.252000000000002
|
|
- type: ndcg_at_5
|
|
value: 27.339000000000002
|
|
- type: precision_at_1
|
|
value: 17.526
|
|
- type: precision_at_10
|
|
value: 5.548
|
|
- type: precision_at_100
|
|
value: 0.88
|
|
- type: precision_at_1000
|
|
value: 0.106
|
|
- type: precision_at_20
|
|
value: 3.3649999999999998
|
|
- type: precision_at_3
|
|
value: 11.25
|
|
- type: precision_at_5
|
|
value: 8.517
|
|
- type: recall_at_1
|
|
value: 15.584999999999999
|
|
- type: recall_at_10
|
|
value: 46.521
|
|
- type: recall_at_100
|
|
value: 72.571
|
|
- type: recall_at_1000
|
|
value: 86.86500000000001
|
|
- type: recall_at_20
|
|
value: 56.004
|
|
- type: recall_at_3
|
|
value: 29.195999999999998
|
|
- type: recall_at_5
|
|
value: 36.324
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB PAC
|
|
revision: fc69d1c153a8ccdcf1eef52f4e2a27f88782f543
|
|
split: test
|
|
type: laugustyniak/abusive-clauses-pl
|
|
metrics:
|
|
- type: accuracy
|
|
value: 64.80162177816392
|
|
- type: ap
|
|
value: 74.10348604798286
|
|
- type: ap_weighted
|
|
value: 74.10348604798286
|
|
- type: f1
|
|
value: 61.280331645723685
|
|
- type: f1_weighted
|
|
value: 65.03859489177282
|
|
- type: main_score
|
|
value: 64.80162177816392
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB PSC
|
|
revision: d05a294af9e1d3ff2bfb6b714e08a24a6cabc669
|
|
split: test
|
|
type: PL-MTEB/psc-pairclassification
|
|
metrics:
|
|
- type: cosine_accuracy
|
|
value: 97.77365491651206
|
|
- type: cosine_accuracy_threshold
|
|
value: 81.08445405960083
|
|
- type: cosine_ap
|
|
value: 99.43195082030653
|
|
- type: cosine_f1
|
|
value: 96.40718562874251
|
|
- type: cosine_f1_threshold
|
|
value: 81.08445405960083
|
|
- type: cosine_precision
|
|
value: 94.70588235294117
|
|
- type: cosine_recall
|
|
value: 98.17073170731707
|
|
- type: dot_accuracy
|
|
value: 97.77365491651206
|
|
- type: dot_accuracy_threshold
|
|
value: 81.08445405960083
|
|
- type: dot_ap
|
|
value: 99.43195082030653
|
|
- type: dot_f1
|
|
value: 96.40718562874251
|
|
- type: dot_f1_threshold
|
|
value: 81.08445405960083
|
|
- type: dot_precision
|
|
value: 94.70588235294117
|
|
- type: dot_recall
|
|
value: 98.17073170731707
|
|
- type: euclidean_accuracy
|
|
value: 97.77365491651206
|
|
- type: euclidean_accuracy_threshold
|
|
value: 61.50695085525513
|
|
- type: euclidean_ap
|
|
value: 99.43195082030653
|
|
- type: euclidean_f1
|
|
value: 96.40718562874251
|
|
- type: euclidean_f1_threshold
|
|
value: 61.50695085525513
|
|
- type: euclidean_precision
|
|
value: 94.70588235294117
|
|
- type: euclidean_recall
|
|
value: 98.17073170731707
|
|
- type: main_score
|
|
value: 99.46339853695966
|
|
- type: manhattan_accuracy
|
|
value: 98.05194805194806
|
|
- type: manhattan_accuracy_threshold
|
|
value: 1428.3578872680664
|
|
- type: manhattan_ap
|
|
value: 99.46339853695966
|
|
- type: manhattan_f1
|
|
value: 96.83257918552036
|
|
- type: manhattan_f1_threshold
|
|
value: 1428.3578872680664
|
|
- type: manhattan_precision
|
|
value: 95.82089552238806
|
|
- type: manhattan_recall
|
|
value: 97.86585365853658
|
|
- type: max_ap
|
|
value: 99.46339853695966
|
|
- type: max_f1
|
|
value: 96.83257918552036
|
|
- type: max_precision
|
|
value: 95.82089552238806
|
|
- type: max_recall
|
|
value: 98.17073170731707
|
|
- type: similarity_accuracy
|
|
value: 97.77365491651206
|
|
- type: similarity_accuracy_threshold
|
|
value: 81.08445405960083
|
|
- type: similarity_ap
|
|
value: 99.43195082030653
|
|
- type: similarity_f1
|
|
value: 96.40718562874251
|
|
- type: similarity_f1_threshold
|
|
value: 81.08445405960083
|
|
- type: similarity_precision
|
|
value: 94.70588235294117
|
|
- type: similarity_recall
|
|
value: 98.17073170731707
|
|
task:
|
|
type: PairClassification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB PolEmo2.0-IN
|
|
revision: d90724373c70959f17d2331ad51fb60c71176b03
|
|
split: test
|
|
type: PL-MTEB/polemo2_in
|
|
metrics:
|
|
- type: accuracy
|
|
value: 80.54016620498614
|
|
- type: f1
|
|
value: 74.07868803329357
|
|
- type: f1_weighted
|
|
value: 78.52375884318697
|
|
- type: main_score
|
|
value: 80.54016620498614
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB PolEmo2.0-OUT
|
|
revision: 6a21ab8716e255ab1867265f8b396105e8aa63d4
|
|
split: test
|
|
type: PL-MTEB/polemo2_out
|
|
metrics:
|
|
- type: accuracy
|
|
value: 61.37651821862349
|
|
- type: f1
|
|
value: 46.60510896853889
|
|
- type: f1_weighted
|
|
value: 61.3956699958363
|
|
- type: main_score
|
|
value: 61.37651821862349
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB PPC
|
|
revision: 2c7d2df57801a591f6b1e3aaf042e7a04ec7d9f2
|
|
split: test
|
|
type: PL-MTEB/ppc-pairclassification
|
|
metrics:
|
|
- type: cosine_accuracy
|
|
value: 76.0
|
|
- type: cosine_accuracy_threshold
|
|
value: 93.07277202606201
|
|
- type: cosine_ap
|
|
value: 87.43755817552731
|
|
- type: cosine_f1
|
|
value: 80.46989720998532
|
|
- type: cosine_f1_threshold
|
|
value: 90.98483324050903
|
|
- type: cosine_precision
|
|
value: 72.29551451187335
|
|
- type: cosine_recall
|
|
value: 90.72847682119205
|
|
- type: dot_accuracy
|
|
value: 76.0
|
|
- type: dot_accuracy_threshold
|
|
value: 93.07277798652649
|
|
- type: dot_ap
|
|
value: 87.43751021710085
|
|
- type: dot_f1
|
|
value: 80.46989720998532
|
|
- type: dot_f1_threshold
|
|
value: 90.98482728004456
|
|
- type: dot_precision
|
|
value: 72.29551451187335
|
|
- type: dot_recall
|
|
value: 90.72847682119205
|
|
- type: euclidean_accuracy
|
|
value: 76.0
|
|
- type: euclidean_accuracy_threshold
|
|
value: 37.221553921699524
|
|
- type: euclidean_ap
|
|
value: 87.43751021710085
|
|
- type: euclidean_f1
|
|
value: 80.46989720998532
|
|
- type: euclidean_f1_threshold
|
|
value: 42.46214032173157
|
|
- type: euclidean_precision
|
|
value: 72.29551451187335
|
|
- type: euclidean_recall
|
|
value: 90.72847682119205
|
|
- type: main_score
|
|
value: 87.43755817552731
|
|
- type: manhattan_accuracy
|
|
value: 75.2
|
|
- type: manhattan_accuracy_threshold
|
|
value: 858.4394454956055
|
|
- type: manhattan_ap
|
|
value: 87.28751334847506
|
|
- type: manhattan_f1
|
|
value: 80.47162859248341
|
|
- type: manhattan_f1_threshold
|
|
value: 981.0188293457031
|
|
- type: manhattan_precision
|
|
value: 72.50996015936255
|
|
- type: manhattan_recall
|
|
value: 90.39735099337747
|
|
- type: max_ap
|
|
value: 87.43755817552731
|
|
- type: max_f1
|
|
value: 80.47162859248341
|
|
- type: max_precision
|
|
value: 72.50996015936255
|
|
- type: max_recall
|
|
value: 90.72847682119205
|
|
- type: similarity_accuracy
|
|
value: 76.0
|
|
- type: similarity_accuracy_threshold
|
|
value: 93.07277202606201
|
|
- type: similarity_ap
|
|
value: 87.43755817552731
|
|
- type: similarity_f1
|
|
value: 80.46989720998532
|
|
- type: similarity_f1_threshold
|
|
value: 90.98483324050903
|
|
- type: similarity_precision
|
|
value: 72.29551451187335
|
|
- type: similarity_recall
|
|
value: 90.72847682119205
|
|
task:
|
|
type: PairClassification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB Quora-PL
|
|
revision: 0be27e93455051e531182b85e85e425aba12e9d4
|
|
split: test
|
|
type: clarin-knext/quora-pl
|
|
metrics:
|
|
- type: main_score
|
|
value: 75.235
|
|
- type: map_at_1
|
|
value: 57.720000000000006
|
|
- type: map_at_10
|
|
value: 70.322
|
|
- type: map_at_100
|
|
value: 71.208
|
|
- type: map_at_1000
|
|
value: 71.247
|
|
- type: map_at_20
|
|
value: 70.889
|
|
- type: map_at_3
|
|
value: 67.278
|
|
- type: map_at_5
|
|
value: 69.07900000000001
|
|
- type: mrr_at_1
|
|
value: 66.44
|
|
- type: mrr_at_10
|
|
value: 74.32428571428532
|
|
- type: mrr_at_100
|
|
value: 74.67001717307676
|
|
- type: mrr_at_1000
|
|
value: 74.68049849872023
|
|
- type: mrr_at_20
|
|
value: 74.55920910032467
|
|
- type: mrr_at_3
|
|
value: 72.6349999999996
|
|
- type: mrr_at_5
|
|
value: 73.67099999999938
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 69.03523613954961
|
|
- type: nauc_map_at_1000_max
|
|
value: 30.29022964222993
|
|
- type: nauc_map_at_1000_std
|
|
value: -13.13676129820498
|
|
- type: nauc_map_at_100_diff1
|
|
value: 69.03918889242972
|
|
- type: nauc_map_at_100_max
|
|
value: 30.28851815152789
|
|
- type: nauc_map_at_100_std
|
|
value: -13.173343854637487
|
|
- type: nauc_map_at_10_diff1
|
|
value: 69.11834037559699
|
|
- type: nauc_map_at_10_max
|
|
value: 29.609089948792388
|
|
- type: nauc_map_at_10_std
|
|
value: -14.511647137697395
|
|
- type: nauc_map_at_1_diff1
|
|
value: 72.50653845898617
|
|
- type: nauc_map_at_1_max
|
|
value: 22.521228683262873
|
|
- type: nauc_map_at_1_std
|
|
value: -17.72541519468729
|
|
- type: nauc_map_at_20_diff1
|
|
value: 69.0572096712263
|
|
- type: nauc_map_at_20_max
|
|
value: 30.09049337817234
|
|
- type: nauc_map_at_20_std
|
|
value: -13.69213787699562
|
|
- type: nauc_map_at_3_diff1
|
|
value: 69.4118549460786
|
|
- type: nauc_map_at_3_max
|
|
value: 27.31606724944123
|
|
- type: nauc_map_at_3_std
|
|
value: -16.430296769671298
|
|
- type: nauc_map_at_5_diff1
|
|
value: 69.18608931793607
|
|
- type: nauc_map_at_5_max
|
|
value: 28.681802217476093
|
|
- type: nauc_map_at_5_std
|
|
value: -15.492619374306827
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 70.27871731978331
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 33.89585229097829
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -9.231498078778678
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 70.27656223213475
|
|
- type: nauc_mrr_at_100_max
|
|
value: 33.90583650980198
|
|
- type: nauc_mrr_at_100_std
|
|
value: -9.213247629622375
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 70.1800255282438
|
|
- type: nauc_mrr_at_10_max
|
|
value: 33.975132933927085
|
|
- type: nauc_mrr_at_10_std
|
|
value: -9.344439026014577
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 72.72425945481199
|
|
- type: nauc_mrr_at_1_max
|
|
value: 31.239650246117385
|
|
- type: nauc_mrr_at_1_std
|
|
value: -11.607242701686696
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 70.24166041655792
|
|
- type: nauc_mrr_at_20_max
|
|
value: 33.9613048334359
|
|
- type: nauc_mrr_at_20_std
|
|
value: -9.219736983314839
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 70.06664104900666
|
|
- type: nauc_mrr_at_3_max
|
|
value: 33.5732140539362
|
|
- type: nauc_mrr_at_3_std
|
|
value: -9.778577982149953
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 70.14739007028493
|
|
- type: nauc_mrr_at_5_max
|
|
value: 33.796518466305834
|
|
- type: nauc_mrr_at_5_std
|
|
value: -9.649151783176043
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 68.62634218438664
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 33.057143795018696
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -9.563352961803663
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 68.58213175533443
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 33.35336572393414
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -9.127811506992467
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 68.26726256015203
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 32.33115112923283
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -11.874276014971688
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 72.66000012395291
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 31.283711202542207
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -11.501503096057867
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 68.39658663907474
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 33.08529095010713
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -10.437492609480433
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 68.05324210316826
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 30.30824001099573
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -13.044199992428771
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 68.10994364753626
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 31.182072802471055
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -12.836057047748234
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -32.848796455727836
|
|
- type: nauc_precision_at_1000_max
|
|
value: 6.715546095139156
|
|
- type: nauc_precision_at_1000_std
|
|
value: 32.9655373056535
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -28.794521134307093
|
|
- type: nauc_precision_at_100_max
|
|
value: 11.155432738297682
|
|
- type: nauc_precision_at_100_std
|
|
value: 33.30986182557851
|
|
- type: nauc_precision_at_10_diff1
|
|
value: -10.613535245108128
|
|
- type: nauc_precision_at_10_max
|
|
value: 19.057316698279582
|
|
- type: nauc_precision_at_10_std
|
|
value: 19.87457963908978
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 72.66000012395291
|
|
- type: nauc_precision_at_1_max
|
|
value: 31.283711202542207
|
|
- type: nauc_precision_at_1_std
|
|
value: -11.501503096057867
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -19.6984185276961
|
|
- type: nauc_precision_at_20_max
|
|
value: 16.497527862287058
|
|
- type: nauc_precision_at_20_std
|
|
value: 26.871607334073012
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 17.130494007304765
|
|
- type: nauc_precision_at_3_max
|
|
value: 23.99199625132106
|
|
- type: nauc_precision_at_3_std
|
|
value: 5.234797091652211
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 3.0202641879085697
|
|
- type: nauc_precision_at_5_max
|
|
value: 22.31257369308076
|
|
- type: nauc_precision_at_5_std
|
|
value: 12.502866671883032
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 49.899967761974196
|
|
- type: nauc_recall_at_1000_max
|
|
value: 54.39990257883846
|
|
- type: nauc_recall_at_1000_std
|
|
value: 42.663306287015196
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 57.87887190551234
|
|
- type: nauc_recall_at_100_max
|
|
value: 48.03395851487758
|
|
- type: nauc_recall_at_100_std
|
|
value: 25.008694604591312
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 60.99359933290845
|
|
- type: nauc_recall_at_10_max
|
|
value: 34.817508290483154
|
|
- type: nauc_recall_at_10_std
|
|
value: -10.355946195658207
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 72.50653845898617
|
|
- type: nauc_recall_at_1_max
|
|
value: 22.521228683262873
|
|
- type: nauc_recall_at_1_std
|
|
value: -17.72541519468729
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 59.63721580389802
|
|
- type: nauc_recall_at_20_max
|
|
value: 39.78324293003396
|
|
- type: nauc_recall_at_20_std
|
|
value: -0.7738431870195353
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 64.28146361759069
|
|
- type: nauc_recall_at_3_max
|
|
value: 27.55821665783294
|
|
- type: nauc_recall_at_3_std
|
|
value: -16.385154477134336
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 62.687585623754046
|
|
- type: nauc_recall_at_5_max
|
|
value: 30.357420406058328
|
|
- type: nauc_recall_at_5_std
|
|
value: -14.95291415876769
|
|
- type: ndcg_at_1
|
|
value: 66.47
|
|
- type: ndcg_at_10
|
|
value: 75.235
|
|
- type: ndcg_at_100
|
|
value: 77.847
|
|
- type: ndcg_at_1000
|
|
value: 78.396
|
|
- type: ndcg_at_20
|
|
value: 76.539
|
|
- type: ndcg_at_3
|
|
value: 71.219
|
|
- type: ndcg_at_5
|
|
value: 73.235
|
|
- type: precision_at_1
|
|
value: 66.47
|
|
- type: precision_at_10
|
|
value: 11.596
|
|
- type: precision_at_100
|
|
value: 1.424
|
|
- type: precision_at_1000
|
|
value: 0.153
|
|
- type: precision_at_20
|
|
value: 6.331
|
|
- type: precision_at_3
|
|
value: 31.130000000000003
|
|
- type: precision_at_5
|
|
value: 20.735999999999997
|
|
- type: recall_at_1
|
|
value: 57.720000000000006
|
|
- type: recall_at_10
|
|
value: 85.249
|
|
- type: recall_at_100
|
|
value: 95.39699999999999
|
|
- type: recall_at_1000
|
|
value: 98.81
|
|
- type: recall_at_20
|
|
value: 89.739
|
|
- type: recall_at_3
|
|
value: 73.978
|
|
- type: recall_at_5
|
|
value: 79.355
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SCIDOCS-PL
|
|
revision: 45452b03f05560207ef19149545f168e596c9337
|
|
split: test
|
|
type: clarin-knext/scidocs-pl
|
|
metrics:
|
|
- type: main_score
|
|
value: 15.174000000000001
|
|
- type: map_at_1
|
|
value: 3.6580000000000004
|
|
- type: map_at_10
|
|
value: 8.796
|
|
- type: map_at_100
|
|
value: 10.391
|
|
- type: map_at_1000
|
|
value: 10.646
|
|
- type: map_at_20
|
|
value: 9.592
|
|
- type: map_at_3
|
|
value: 6.489000000000001
|
|
- type: map_at_5
|
|
value: 7.600999999999999
|
|
- type: mrr_at_1
|
|
value: 18.0
|
|
- type: mrr_at_10
|
|
value: 26.845317460317457
|
|
- type: mrr_at_100
|
|
value: 28.04995949015167
|
|
- type: mrr_at_1000
|
|
value: 28.121893269944824
|
|
- type: mrr_at_20
|
|
value: 27.566026091211864
|
|
- type: mrr_at_3
|
|
value: 23.916666666666686
|
|
- type: mrr_at_5
|
|
value: 25.551666666666666
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 17.302827041650488
|
|
- type: nauc_map_at_1000_max
|
|
value: 26.65992706695422
|
|
- type: nauc_map_at_1000_std
|
|
value: 18.96964501922404
|
|
- type: nauc_map_at_100_diff1
|
|
value: 17.21226432890004
|
|
- type: nauc_map_at_100_max
|
|
value: 26.45824637348571
|
|
- type: nauc_map_at_100_std
|
|
value: 18.573352847100065
|
|
- type: nauc_map_at_10_diff1
|
|
value: 17.02056023363081
|
|
- type: nauc_map_at_10_max
|
|
value: 24.48428170985602
|
|
- type: nauc_map_at_10_std
|
|
value: 14.014378375804235
|
|
- type: nauc_map_at_1_diff1
|
|
value: 21.638506619768716
|
|
- type: nauc_map_at_1_max
|
|
value: 19.709230810058283
|
|
- type: nauc_map_at_1_std
|
|
value: 9.042419739024966
|
|
- type: nauc_map_at_20_diff1
|
|
value: 17.067893569553323
|
|
- type: nauc_map_at_20_max
|
|
value: 25.69106547536296
|
|
- type: nauc_map_at_20_std
|
|
value: 16.535327068913993
|
|
- type: nauc_map_at_3_diff1
|
|
value: 18.56349850011108
|
|
- type: nauc_map_at_3_max
|
|
value: 22.127177599224744
|
|
- type: nauc_map_at_3_std
|
|
value: 9.47260767358392
|
|
- type: nauc_map_at_5_diff1
|
|
value: 18.05585009830461
|
|
- type: nauc_map_at_5_max
|
|
value: 23.31477343090323
|
|
- type: nauc_map_at_5_std
|
|
value: 11.257936348356862
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 19.71318833342125
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 22.359300102570092
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 13.89561747692388
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 19.709804653242603
|
|
- type: nauc_mrr_at_100_max
|
|
value: 22.365551370687967
|
|
- type: nauc_mrr_at_100_std
|
|
value: 13.918573803759068
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 19.74677273038544
|
|
- type: nauc_mrr_at_10_max
|
|
value: 22.348783997030335
|
|
- type: nauc_mrr_at_10_std
|
|
value: 13.606175345418963
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 21.957688664351128
|
|
- type: nauc_mrr_at_1_max
|
|
value: 19.50356102866365
|
|
- type: nauc_mrr_at_1_std
|
|
value: 9.323755394169037
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 19.5076818806823
|
|
- type: nauc_mrr_at_20_max
|
|
value: 22.192342439483934
|
|
- type: nauc_mrr_at_20_std
|
|
value: 13.705438410110608
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 19.784830140193804
|
|
- type: nauc_mrr_at_3_max
|
|
value: 21.606746947165416
|
|
- type: nauc_mrr_at_3_std
|
|
value: 12.289045699872666
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 20.139962218896674
|
|
- type: nauc_mrr_at_5_max
|
|
value: 22.139813460789266
|
|
- type: nauc_mrr_at_5_std
|
|
value: 13.177813432176084
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 17.78059204124948
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 29.830544327132436
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 28.03254237837783
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 17.62481104076364
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 28.629131876483665
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 26.019853664301124
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 17.25237540570343
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 25.128032787033604
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 16.571629975349868
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 21.957688664351128
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 19.50356102866365
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 9.323755394169037
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 16.549388210526494
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 26.1871953370256
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 19.971064555030125
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 18.707127276019474
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 22.042786711511813
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 11.103829353868623
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 18.45321448876598
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 23.475902453066492
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 13.216222368946411
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 11.843768977161584
|
|
- type: nauc_precision_at_1000_max
|
|
value: 30.300299347010352
|
|
- type: nauc_precision_at_1000_std
|
|
value: 41.123748924498585
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 13.765676375073074
|
|
- type: nauc_precision_at_100_max
|
|
value: 29.769561801824956
|
|
- type: nauc_precision_at_100_std
|
|
value: 37.56343888054612
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 14.123009605345343
|
|
- type: nauc_precision_at_10_max
|
|
value: 26.045793706986558
|
|
- type: nauc_precision_at_10_std
|
|
value: 20.45802977436883
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 21.957688664351128
|
|
- type: nauc_precision_at_1_max
|
|
value: 19.50356102866365
|
|
- type: nauc_precision_at_1_std
|
|
value: 9.323755394169037
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 12.080580953868749
|
|
- type: nauc_precision_at_20_max
|
|
value: 26.741203934729374
|
|
- type: nauc_precision_at_20_std
|
|
value: 26.249289307014976
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 17.390833784290034
|
|
- type: nauc_precision_at_3_max
|
|
value: 22.639415005064585
|
|
- type: nauc_precision_at_3_std
|
|
value: 11.481404394862311
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 17.18007614612505
|
|
- type: nauc_precision_at_5_max
|
|
value: 24.244045184229563
|
|
- type: nauc_precision_at_5_std
|
|
value: 15.180528647694574
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 11.507406580463488
|
|
- type: nauc_recall_at_1000_max
|
|
value: 30.78976497232251
|
|
- type: nauc_recall_at_1000_std
|
|
value: 41.618419379918855
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 13.408507737517144
|
|
- type: nauc_recall_at_100_max
|
|
value: 29.849796157178197
|
|
- type: nauc_recall_at_100_std
|
|
value: 37.58778281760627
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 13.942112101503866
|
|
- type: nauc_recall_at_10_max
|
|
value: 26.228452951171487
|
|
- type: nauc_recall_at_10_std
|
|
value: 20.14835260352246
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 21.638506619768716
|
|
- type: nauc_recall_at_1_max
|
|
value: 19.709230810058283
|
|
- type: nauc_recall_at_1_std
|
|
value: 9.042419739024966
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 11.905542570350702
|
|
- type: nauc_recall_at_20_max
|
|
value: 26.84107459006622
|
|
- type: nauc_recall_at_20_std
|
|
value: 25.888986621614645
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 17.056201299401692
|
|
- type: nauc_recall_at_3_max
|
|
value: 22.94288018834461
|
|
- type: nauc_recall_at_3_std
|
|
value: 11.337560544201224
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 16.89022137209632
|
|
- type: nauc_recall_at_5_max
|
|
value: 24.564195711081545
|
|
- type: nauc_recall_at_5_std
|
|
value: 14.979769166201622
|
|
- type: ndcg_at_1
|
|
value: 18.0
|
|
- type: ndcg_at_10
|
|
value: 15.174000000000001
|
|
- type: ndcg_at_100
|
|
value: 22.047
|
|
- type: ndcg_at_1000
|
|
value: 27.057
|
|
- type: ndcg_at_20
|
|
value: 17.628
|
|
- type: ndcg_at_3
|
|
value: 14.536999999999999
|
|
- type: ndcg_at_5
|
|
value: 12.590000000000002
|
|
- type: precision_at_1
|
|
value: 18.0
|
|
- type: precision_at_10
|
|
value: 7.82
|
|
- type: precision_at_100
|
|
value: 1.773
|
|
- type: precision_at_1000
|
|
value: 0.298
|
|
- type: precision_at_20
|
|
value: 5.335
|
|
- type: precision_at_3
|
|
value: 13.5
|
|
- type: precision_at_5
|
|
value: 10.92
|
|
- type: recall_at_1
|
|
value: 3.6580000000000004
|
|
- type: recall_at_10
|
|
value: 15.867999999999999
|
|
- type: recall_at_100
|
|
value: 36.068
|
|
- type: recall_at_1000
|
|
value: 60.608
|
|
- type: recall_at_20
|
|
value: 21.653
|
|
- type: recall_at_3
|
|
value: 8.248
|
|
- type: recall_at_5
|
|
value: 11.108
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SICK-E-PL
|
|
revision: 71bba34b0ece6c56dfcf46d9758a27f7a90f17e9
|
|
split: test
|
|
type: PL-MTEB/sicke-pl-pairclassification
|
|
metrics:
|
|
- type: cosine_accuracy
|
|
value: 78.12882185079495
|
|
- type: cosine_accuracy_threshold
|
|
value: 95.76345682144165
|
|
- type: cosine_ap
|
|
value: 63.56538407363026
|
|
- type: cosine_f1
|
|
value: 60.88388690639582
|
|
- type: cosine_f1_threshold
|
|
value: 92.86266565322876
|
|
- type: cosine_precision
|
|
value: 49.53104064314426
|
|
- type: cosine_recall
|
|
value: 78.98860398860398
|
|
- type: dot_accuracy
|
|
value: 78.12882185079495
|
|
- type: dot_accuracy_threshold
|
|
value: 95.76345682144165
|
|
- type: dot_ap
|
|
value: 63.56553287602377
|
|
- type: dot_f1
|
|
value: 60.88388690639582
|
|
- type: dot_f1_threshold
|
|
value: 92.86266565322876
|
|
- type: dot_precision
|
|
value: 49.53104064314426
|
|
- type: dot_recall
|
|
value: 78.98860398860398
|
|
- type: euclidean_accuracy
|
|
value: 78.12882185079495
|
|
- type: euclidean_accuracy_threshold
|
|
value: 29.108554124832153
|
|
- type: euclidean_ap
|
|
value: 63.56543484315041
|
|
- type: euclidean_f1
|
|
value: 60.88388690639582
|
|
- type: euclidean_f1_threshold
|
|
value: 37.781822681427
|
|
- type: euclidean_precision
|
|
value: 49.53104064314426
|
|
- type: euclidean_recall
|
|
value: 78.98860398860398
|
|
- type: main_score
|
|
value: 63.56553287602377
|
|
- type: manhattan_accuracy
|
|
value: 77.82307378719935
|
|
- type: manhattan_accuracy_threshold
|
|
value: 658.8656902313232
|
|
- type: manhattan_ap
|
|
value: 63.12761769067177
|
|
- type: manhattan_f1
|
|
value: 60.76436623590872
|
|
- type: manhattan_f1_threshold
|
|
value: 888.3136749267578
|
|
- type: manhattan_precision
|
|
value: 49.48499776085983
|
|
- type: manhattan_recall
|
|
value: 78.70370370370371
|
|
- type: max_ap
|
|
value: 63.56553287602377
|
|
- type: max_f1
|
|
value: 60.88388690639582
|
|
- type: max_precision
|
|
value: 49.53104064314426
|
|
- type: max_recall
|
|
value: 78.98860398860398
|
|
- type: similarity_accuracy
|
|
value: 78.12882185079495
|
|
- type: similarity_accuracy_threshold
|
|
value: 95.76345682144165
|
|
- type: similarity_ap
|
|
value: 63.56538407363026
|
|
- type: similarity_f1
|
|
value: 60.88388690639582
|
|
- type: similarity_f1_threshold
|
|
value: 92.86266565322876
|
|
- type: similarity_precision
|
|
value: 49.53104064314426
|
|
- type: similarity_recall
|
|
value: 78.98860398860398
|
|
task:
|
|
type: PairClassification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SICK-R-PL
|
|
revision: fd5c2441b7eeff8676768036142af4cfa42c1339
|
|
split: test
|
|
type: PL-MTEB/sickr-pl-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 71.75502609028113
|
|
- type: cosine_spearman
|
|
value: 66.52097638938338
|
|
- type: euclidean_pearson
|
|
value: 68.6974439167054
|
|
- type: euclidean_spearman
|
|
value: 66.52095939114172
|
|
- type: main_score
|
|
value: 66.52097638938338
|
|
- type: manhattan_pearson
|
|
value: 68.53848708135571
|
|
- type: manhattan_spearman
|
|
value: 66.29909223435631
|
|
- type: pearson
|
|
value: 71.75502609028113
|
|
- type: spearman
|
|
value: 66.52097638938338
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: pl
|
|
name: MTEB STS22 (pl)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 40.06621394078099
|
|
- type: cosine_spearman
|
|
value: 45.160446103264285
|
|
- type: euclidean_pearson
|
|
value: 25.38908314629843
|
|
- type: euclidean_spearman
|
|
value: 45.160446103264285
|
|
- type: main_score
|
|
value: 45.160446103264285
|
|
- type: manhattan_pearson
|
|
value: 25.13217941116968
|
|
- type: manhattan_spearman
|
|
value: 45.05397285684081
|
|
- type: pearson
|
|
value: 40.06621394078099
|
|
- type: spearman
|
|
value: 45.160446103264285
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: pl-en
|
|
name: MTEB STS22 (pl-en)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 77.99167158750629
|
|
- type: cosine_spearman
|
|
value: 77.00326330683939
|
|
- type: euclidean_pearson
|
|
value: 77.60571751826936
|
|
- type: euclidean_spearman
|
|
value: 77.00326330683939
|
|
- type: main_score
|
|
value: 77.00326330683939
|
|
- type: manhattan_pearson
|
|
value: 78.19839585217989
|
|
- type: manhattan_spearman
|
|
value: 78.44894390841364
|
|
- type: pearson
|
|
value: 77.99167158750629
|
|
- type: spearman
|
|
value: 77.00326330683939
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: de-pl
|
|
name: MTEB STS22 (de-pl)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 40.2221719679774
|
|
- type: cosine_spearman
|
|
value: 57.18465019880842
|
|
- type: euclidean_pearson
|
|
value: 42.11211158455479
|
|
- type: euclidean_spearman
|
|
value: 57.18465019880842
|
|
- type: main_score
|
|
value: 57.18465019880842
|
|
- type: manhattan_pearson
|
|
value: 43.24148614152723
|
|
- type: manhattan_spearman
|
|
value: 56.35320940431847
|
|
- type: pearson
|
|
value: 40.2221719679774
|
|
- type: spearman
|
|
value: 57.18465019880842
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: fr-pl
|
|
name: MTEB STS22 (fr-pl)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 88.17138238483673
|
|
- type: cosine_spearman
|
|
value: 84.51542547285167
|
|
- type: euclidean_pearson
|
|
value: 87.99782696047525
|
|
- type: euclidean_spearman
|
|
value: 84.51542547285167
|
|
- type: main_score
|
|
value: 84.51542547285167
|
|
- type: manhattan_pearson
|
|
value: 85.811937669563
|
|
- type: manhattan_spearman
|
|
value: 84.51542547285167
|
|
- type: pearson
|
|
value: 88.17138238483673
|
|
- type: spearman
|
|
value: 84.51542547285167
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SciFact-PL
|
|
revision: 47932a35f045ef8ed01ba82bf9ff67f6e109207e
|
|
split: test
|
|
type: clarin-knext/scifact-pl
|
|
metrics:
|
|
- type: main_score
|
|
value: 62.064
|
|
- type: map_at_1
|
|
value: 48.317
|
|
- type: map_at_10
|
|
value: 57.693000000000005
|
|
- type: map_at_100
|
|
value: 58.392999999999994
|
|
- type: map_at_1000
|
|
value: 58.428999999999995
|
|
- type: map_at_20
|
|
value: 58.108000000000004
|
|
- type: map_at_3
|
|
value: 55.293000000000006
|
|
- type: map_at_5
|
|
value: 56.595
|
|
- type: mrr_at_1
|
|
value: 51.0
|
|
- type: mrr_at_10
|
|
value: 59.019576719576705
|
|
- type: mrr_at_100
|
|
value: 59.58007358566797
|
|
- type: mrr_at_1000
|
|
value: 59.61403985820887
|
|
- type: mrr_at_20
|
|
value: 59.35199007075942
|
|
- type: mrr_at_3
|
|
value: 57.166666666666664
|
|
- type: mrr_at_5
|
|
value: 58.08333333333332
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 55.90310480193163
|
|
- type: nauc_map_at_1000_max
|
|
value: 40.922646499130586
|
|
- type: nauc_map_at_1000_std
|
|
value: 6.308307542867231
|
|
- type: nauc_map_at_100_diff1
|
|
value: 55.87923016501095
|
|
- type: nauc_map_at_100_max
|
|
value: 40.930429212300396
|
|
- type: nauc_map_at_100_std
|
|
value: 6.302652510324859
|
|
- type: nauc_map_at_10_diff1
|
|
value: 55.96811326806582
|
|
- type: nauc_map_at_10_max
|
|
value: 40.91912121040118
|
|
- type: nauc_map_at_10_std
|
|
value: 6.315081020792943
|
|
- type: nauc_map_at_1_diff1
|
|
value: 61.615316460538374
|
|
- type: nauc_map_at_1_max
|
|
value: 34.4312789344494
|
|
- type: nauc_map_at_1_std
|
|
value: -2.151749018851701
|
|
- type: nauc_map_at_20_diff1
|
|
value: 55.781940594193316
|
|
- type: nauc_map_at_20_max
|
|
value: 40.877518039008585
|
|
- type: nauc_map_at_20_std
|
|
value: 6.170527123248918
|
|
- type: nauc_map_at_3_diff1
|
|
value: 58.104315292507216
|
|
- type: nauc_map_at_3_max
|
|
value: 39.524635028616544
|
|
- type: nauc_map_at_3_std
|
|
value: 4.367263811245541
|
|
- type: nauc_map_at_5_diff1
|
|
value: 56.60725686218003
|
|
- type: nauc_map_at_5_max
|
|
value: 40.362341129747456
|
|
- type: nauc_map_at_5_std
|
|
value: 5.222556427559299
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 56.243518111487454
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 41.92306224416779
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 7.331011181148979
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 56.21745814714038
|
|
- type: nauc_mrr_at_100_max
|
|
value: 41.92847851363498
|
|
- type: nauc_mrr_at_100_std
|
|
value: 7.322136402819359
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 56.22224221410973
|
|
- type: nauc_mrr_at_10_max
|
|
value: 42.020110225540144
|
|
- type: nauc_mrr_at_10_std
|
|
value: 7.367785001729785
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 61.65968884760533
|
|
- type: nauc_mrr_at_1_max
|
|
value: 39.22611274899148
|
|
- type: nauc_mrr_at_1_std
|
|
value: 3.3484556807524357
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 56.140226618395495
|
|
- type: nauc_mrr_at_20_max
|
|
value: 41.92506913405156
|
|
- type: nauc_mrr_at_20_std
|
|
value: 7.20339996949852
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 57.82506573973446
|
|
- type: nauc_mrr_at_3_max
|
|
value: 41.962001263558484
|
|
- type: nauc_mrr_at_3_std
|
|
value: 6.909954113302328
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 56.659054585223565
|
|
- type: nauc_mrr_at_5_max
|
|
value: 42.220145330498326
|
|
- type: nauc_mrr_at_5_std
|
|
value: 6.914754115832333
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 54.101423320176956
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 42.35761455565217
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 9.158968107515042
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 53.193377266960695
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 42.39818084789296
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 8.982680006715663
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 52.7521864873992
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 42.25954681169497
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 9.025856795668409
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 61.65968884760533
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 39.22611274899148
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 3.3484556807524357
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 52.24054304553779
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 42.14484844258701
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 8.522811774790046
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 56.65801023652111
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 41.59901000744857
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 6.866411754213651
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 54.25032835371862
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 41.52568005051319
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 6.747184564934237
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -12.438995870489618
|
|
- type: nauc_precision_at_1000_max
|
|
value: 33.65458584888833
|
|
- type: nauc_precision_at_1000_std
|
|
value: 38.65000092313945
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -3.7051397832573696
|
|
- type: nauc_precision_at_100_max
|
|
value: 36.777033924925384
|
|
- type: nauc_precision_at_100_std
|
|
value: 32.24732998272339
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 14.458974499542448
|
|
- type: nauc_precision_at_10_max
|
|
value: 45.75828754327736
|
|
- type: nauc_precision_at_10_std
|
|
value: 31.734511856215665
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 61.65968884760533
|
|
- type: nauc_precision_at_1_max
|
|
value: 39.22611274899148
|
|
- type: nauc_precision_at_1_std
|
|
value: 3.3484556807524357
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 6.911000226020142
|
|
- type: nauc_precision_at_20_max
|
|
value: 42.75953196446269
|
|
- type: nauc_precision_at_20_std
|
|
value: 30.293217657388254
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 39.95888414475174
|
|
- type: nauc_precision_at_3_max
|
|
value: 46.81095681980396
|
|
- type: nauc_precision_at_3_std
|
|
value: 20.732734118894037
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 27.25227607416867
|
|
- type: nauc_precision_at_5_max
|
|
value: 45.278620768210615
|
|
- type: nauc_precision_at_5_std
|
|
value: 22.7094842525771
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 54.66853408029846
|
|
- type: nauc_recall_at_1000_max
|
|
value: 69.49112978524705
|
|
- type: nauc_recall_at_1000_std
|
|
value: 84.76890756302552
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 33.641140071848085
|
|
- type: nauc_recall_at_100_max
|
|
value: 49.94619316653212
|
|
- type: nauc_recall_at_100_std
|
|
value: 26.970675275760104
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 38.56340942303001
|
|
- type: nauc_recall_at_10_max
|
|
value: 44.13889679913801
|
|
- type: nauc_recall_at_10_std
|
|
value: 17.814455740104584
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 61.615316460538374
|
|
- type: nauc_recall_at_1_max
|
|
value: 34.4312789344494
|
|
- type: nauc_recall_at_1_std
|
|
value: -2.151749018851701
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 33.86997626483988
|
|
- type: nauc_recall_at_20_max
|
|
value: 44.31136705663488
|
|
- type: nauc_recall_at_20_std
|
|
value: 16.58271492635832
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 52.39739118413791
|
|
- type: nauc_recall_at_3_max
|
|
value: 40.56472420414715
|
|
- type: nauc_recall_at_3_std
|
|
value: 7.856902134348368
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 45.693766776717595
|
|
- type: nauc_recall_at_5_max
|
|
value: 41.817545551209086
|
|
- type: nauc_recall_at_5_std
|
|
value: 9.066813773598692
|
|
- type: ndcg_at_1
|
|
value: 51.0
|
|
- type: ndcg_at_10
|
|
value: 62.064
|
|
- type: ndcg_at_100
|
|
value: 65.45
|
|
- type: ndcg_at_1000
|
|
value: 66.366
|
|
- type: ndcg_at_20
|
|
value: 63.418
|
|
- type: ndcg_at_3
|
|
value: 57.915000000000006
|
|
- type: ndcg_at_5
|
|
value: 59.65200000000001
|
|
- type: precision_at_1
|
|
value: 51.0
|
|
- type: precision_at_10
|
|
value: 8.433
|
|
- type: precision_at_100
|
|
value: 1.03
|
|
- type: precision_at_1000
|
|
value: 0.11
|
|
- type: precision_at_20
|
|
value: 4.517
|
|
- type: precision_at_3
|
|
value: 23.0
|
|
- type: precision_at_5
|
|
value: 15.067
|
|
- type: recall_at_1
|
|
value: 48.317
|
|
- type: recall_at_10
|
|
value: 74.078
|
|
- type: recall_at_100
|
|
value: 90.167
|
|
- type: recall_at_1000
|
|
value: 97.333
|
|
- type: recall_at_20
|
|
value: 79.256
|
|
- type: recall_at_3
|
|
value: 62.561
|
|
- type: recall_at_5
|
|
value: 67.039
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB TRECCOVID-PL
|
|
revision: 81bcb408f33366c2a20ac54adafad1ae7e877fdd
|
|
split: test
|
|
type: clarin-knext/trec-covid-pl
|
|
metrics:
|
|
- type: main_score
|
|
value: 69.244
|
|
- type: map_at_1
|
|
value: 0.216
|
|
- type: map_at_10
|
|
value: 1.717
|
|
- type: map_at_100
|
|
value: 9.051
|
|
- type: map_at_1000
|
|
value: 21.688
|
|
- type: map_at_20
|
|
value: 2.972
|
|
- type: map_at_3
|
|
value: 0.624
|
|
- type: map_at_5
|
|
value: 0.9809999999999999
|
|
- type: mrr_at_1
|
|
value: 82.0
|
|
- type: mrr_at_10
|
|
value: 88.41666666666666
|
|
- type: mrr_at_100
|
|
value: 88.57051282051282
|
|
- type: mrr_at_1000
|
|
value: 88.57051282051282
|
|
- type: mrr_at_20
|
|
value: 88.57051282051282
|
|
- type: mrr_at_3
|
|
value: 87.66666666666666
|
|
- type: mrr_at_5
|
|
value: 88.16666666666666
|
|
- type: nauc_map_at_1000_diff1
|
|
value: -21.210172839828886
|
|
- type: nauc_map_at_1000_max
|
|
value: 50.364439193708456
|
|
- type: nauc_map_at_1000_std
|
|
value: 82.23413161215711
|
|
- type: nauc_map_at_100_diff1
|
|
value: -3.737989437317314
|
|
- type: nauc_map_at_100_max
|
|
value: 40.24314095187729
|
|
- type: nauc_map_at_100_std
|
|
value: 74.6556355692718
|
|
- type: nauc_map_at_10_diff1
|
|
value: 24.069758586207186
|
|
- type: nauc_map_at_10_max
|
|
value: 25.978576944212445
|
|
- type: nauc_map_at_10_std
|
|
value: 30.92185789388276
|
|
- type: nauc_map_at_1_diff1
|
|
value: 33.44422662406722
|
|
- type: nauc_map_at_1_max
|
|
value: 18.58849173002632
|
|
- type: nauc_map_at_1_std
|
|
value: 23.001195148863555
|
|
- type: nauc_map_at_20_diff1
|
|
value: 16.195748164952704
|
|
- type: nauc_map_at_20_max
|
|
value: 32.418991157208055
|
|
- type: nauc_map_at_20_std
|
|
value: 45.053299350375795
|
|
- type: nauc_map_at_3_diff1
|
|
value: 32.94899528110181
|
|
- type: nauc_map_at_3_max
|
|
value: 16.721379232494304
|
|
- type: nauc_map_at_3_std
|
|
value: 18.336699336799814
|
|
- type: nauc_map_at_5_diff1
|
|
value: 30.34930846309755
|
|
- type: nauc_map_at_5_max
|
|
value: 19.37661209832802
|
|
- type: nauc_map_at_5_std
|
|
value: 20.312897662543314
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 49.418158929182006
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 67.05328023364747
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 70.85520896614209
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 49.418158929182006
|
|
- type: nauc_mrr_at_100_max
|
|
value: 67.05328023364747
|
|
- type: nauc_mrr_at_100_std
|
|
value: 70.85520896614209
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 49.50157932873256
|
|
- type: nauc_mrr_at_10_max
|
|
value: 65.88227845429796
|
|
- type: nauc_mrr_at_10_std
|
|
value: 70.87422352601853
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 44.82872563057607
|
|
- type: nauc_mrr_at_1_max
|
|
value: 70.45930168520755
|
|
- type: nauc_mrr_at_1_std
|
|
value: 69.88104416785988
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 49.418158929182006
|
|
- type: nauc_mrr_at_20_max
|
|
value: 67.05328023364747
|
|
- type: nauc_mrr_at_20_std
|
|
value: 70.85520896614209
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 49.71407489393107
|
|
- type: nauc_mrr_at_3_max
|
|
value: 67.77215590165227
|
|
- type: nauc_mrr_at_3_std
|
|
value: 72.72379898279185
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 50.328834220772976
|
|
- type: nauc_mrr_at_5_max
|
|
value: 66.34746357369875
|
|
- type: nauc_mrr_at_5_std
|
|
value: 71.51800332961842
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: -11.723371568664843
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 53.41150083076567
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 81.94372023908832
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: -15.990454633114279
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 45.35431514782352
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 75.73014493320755
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 4.30050518239422
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 50.83631607203189
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 63.1087699434136
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 17.206529677661354
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 62.14050255620695
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 64.51116243264046
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: -5.9182205607515685
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 49.12802457140552
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 68.77672262568693
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 22.158007969692125
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 48.17593837968984
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 58.4991887813489
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 16.89487399786786
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 46.752900245009414
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 60.870638593862914
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -24.67751088399524
|
|
- type: nauc_precision_at_1000_max
|
|
value: 42.70887481946044
|
|
- type: nauc_precision_at_1000_std
|
|
value: 49.219386318590566
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -19.829901963316278
|
|
- type: nauc_precision_at_100_max
|
|
value: 44.4613898680245
|
|
- type: nauc_precision_at_100_std
|
|
value: 74.8829067578589
|
|
- type: nauc_precision_at_10_diff1
|
|
value: -0.6759004971171398
|
|
- type: nauc_precision_at_10_max
|
|
value: 52.16154071543153
|
|
- type: nauc_precision_at_10_std
|
|
value: 62.98886080224083
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 44.82872563057607
|
|
- type: nauc_precision_at_1_max
|
|
value: 70.45930168520755
|
|
- type: nauc_precision_at_1_std
|
|
value: 69.88104416785988
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -11.458671607862547
|
|
- type: nauc_precision_at_20_max
|
|
value: 49.71202888307331
|
|
- type: nauc_precision_at_20_std
|
|
value: 71.79100842422972
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 30.23048096153466
|
|
- type: nauc_precision_at_3_max
|
|
value: 48.24954855245538
|
|
- type: nauc_precision_at_3_std
|
|
value: 54.344575833478935
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 13.925893655561437
|
|
- type: nauc_precision_at_5_max
|
|
value: 46.23506752573775
|
|
- type: nauc_precision_at_5_std
|
|
value: 59.610666544378944
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: -13.691809447793393
|
|
- type: nauc_recall_at_1000_max
|
|
value: 50.39633577248049
|
|
- type: nauc_recall_at_1000_std
|
|
value: 76.65225154588104
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 4.67778695632382
|
|
- type: nauc_recall_at_100_max
|
|
value: 30.19071079451134
|
|
- type: nauc_recall_at_100_std
|
|
value: 65.03682595699173
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 26.24600831247693
|
|
- type: nauc_recall_at_10_max
|
|
value: 22.235399614875632
|
|
- type: nauc_recall_at_10_std
|
|
value: 27.653841671594176
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 33.44422662406722
|
|
- type: nauc_recall_at_1_max
|
|
value: 18.58849173002632
|
|
- type: nauc_recall_at_1_std
|
|
value: 23.001195148863555
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 19.13211263378722
|
|
- type: nauc_recall_at_20_max
|
|
value: 26.697525172621827
|
|
- type: nauc_recall_at_20_std
|
|
value: 40.9095035359023
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 30.47343886364865
|
|
- type: nauc_recall_at_3_max
|
|
value: 12.854379330237647
|
|
- type: nauc_recall_at_3_std
|
|
value: 14.711252261798258
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 28.344400535065112
|
|
- type: nauc_recall_at_5_max
|
|
value: 14.755638630484144
|
|
- type: nauc_recall_at_5_std
|
|
value: 15.864031786019787
|
|
- type: ndcg_at_1
|
|
value: 72.0
|
|
- type: ndcg_at_10
|
|
value: 69.244
|
|
- type: ndcg_at_100
|
|
value: 50.834
|
|
- type: ndcg_at_1000
|
|
value: 45.535
|
|
- type: ndcg_at_20
|
|
value: 65.676
|
|
- type: ndcg_at_3
|
|
value: 73.776
|
|
- type: ndcg_at_5
|
|
value: 72.715
|
|
- type: precision_at_1
|
|
value: 82.0
|
|
- type: precision_at_10
|
|
value: 73.6
|
|
- type: precision_at_100
|
|
value: 52.22
|
|
- type: precision_at_1000
|
|
value: 20.380000000000003
|
|
- type: precision_at_20
|
|
value: 69.0
|
|
- type: precision_at_3
|
|
value: 81.333
|
|
- type: precision_at_5
|
|
value: 79.2
|
|
- type: recall_at_1
|
|
value: 0.216
|
|
- type: recall_at_10
|
|
value: 1.8900000000000001
|
|
- type: recall_at_100
|
|
value: 12.359
|
|
- type: recall_at_1000
|
|
value: 42.791000000000004
|
|
- type: recall_at_20
|
|
value: 3.44
|
|
- type: recall_at_3
|
|
value: 0.653
|
|
- type: recall_at_5
|
|
value: 1.048
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB CEDRClassification
|
|
revision: c0ba03d058e3e1b2f3fd20518875a4563dd12db4
|
|
split: test
|
|
type: ai-forever/cedr-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 43.29968119022317
|
|
- type: f1
|
|
value: 41.112000768648386
|
|
- type: lrap
|
|
value: 72.06216790648348
|
|
- type: main_score
|
|
value: 43.29968119022317
|
|
task:
|
|
type: MultilabelClassification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB GeoreviewClassification
|
|
revision: 3765c0d1de6b7d264bc459433c45e5a75513839c
|
|
split: test
|
|
type: ai-forever/georeview-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 52.0361328125
|
|
- type: f1
|
|
value: 47.84397823612054
|
|
- type: f1_weighted
|
|
value: 47.84111706041435
|
|
- type: main_score
|
|
value: 52.0361328125
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB GeoreviewClusteringP2P
|
|
revision: 97a313c8fc85b47f13f33e7e9a95c1ad888c7fec
|
|
split: test
|
|
type: ai-forever/georeview-clustering-p2p
|
|
metrics:
|
|
- type: main_score
|
|
value: 60.28266888390485
|
|
- type: v_measure
|
|
value: 60.28266888390485
|
|
- type: v_measure_std
|
|
value: 1.0348363132473835
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB HeadlineClassification
|
|
revision: 2fe05ee6b5832cda29f2ef7aaad7b7fe6a3609eb
|
|
split: test
|
|
type: ai-forever/headline-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 83.4033203125
|
|
- type: f1
|
|
value: 83.39708551274371
|
|
- type: f1_weighted
|
|
value: 83.39502222187862
|
|
- type: main_score
|
|
value: 83.4033203125
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB InappropriatenessClassification
|
|
revision: 601651fdc45ef243751676e62dd7a19f491c0285
|
|
split: test
|
|
type: ai-forever/inappropriateness-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 64.140625
|
|
- type: ap
|
|
value: 59.28880813167948
|
|
- type: ap_weighted
|
|
value: 59.28880813167948
|
|
- type: f1
|
|
value: 63.72032598814496
|
|
- type: f1_weighted
|
|
value: 63.72032598814496
|
|
- type: main_score
|
|
value: 64.140625
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB KinopoiskClassification
|
|
revision: 5911f26666ac11af46cb9c6849d0dc80a378af24
|
|
split: test
|
|
type: ai-forever/kinopoisk-sentiment-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 63.15333333333333
|
|
- type: f1
|
|
value: 59.395986541732384
|
|
- type: f1_weighted
|
|
value: 59.395986541732384
|
|
- type: main_score
|
|
value: 63.15333333333333
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: ru
|
|
name: MTEB MIRACLReranking (ru)
|
|
revision: 6d1962c527217f8927fca80f890f14f36b2802af
|
|
split: dev
|
|
type: miracl/mmteb-miracl-reranking
|
|
metrics:
|
|
- type: MAP@1(MIRACL)
|
|
value: 29.732999999999997
|
|
- type: MAP@10(MIRACL)
|
|
value: 48.333
|
|
- type: MAP@100(MIRACL)
|
|
value: 50.517
|
|
- type: MAP@1000(MIRACL)
|
|
value: 50.517
|
|
- type: MAP@20(MIRACL)
|
|
value: 49.85
|
|
- type: MAP@3(MIRACL)
|
|
value: 41.843
|
|
- type: MAP@5(MIRACL)
|
|
value: 45.323
|
|
- type: NDCG@1(MIRACL)
|
|
value: 48.436
|
|
- type: NDCG@10(MIRACL)
|
|
value: 56.111999999999995
|
|
- type: NDCG@100(MIRACL)
|
|
value: 60.617
|
|
- type: NDCG@1000(MIRACL)
|
|
value: 60.617
|
|
- type: NDCG@20(MIRACL)
|
|
value: 58.826
|
|
- type: NDCG@3(MIRACL)
|
|
value: 50.483999999999995
|
|
- type: NDCG@5(MIRACL)
|
|
value: 52.61
|
|
- type: P@1(MIRACL)
|
|
value: 48.436
|
|
- type: P@10(MIRACL)
|
|
value: 14.667
|
|
- type: P@100(MIRACL)
|
|
value: 1.9529999999999998
|
|
- type: P@1000(MIRACL)
|
|
value: 0.19499999999999998
|
|
- type: P@20(MIRACL)
|
|
value: 8.665000000000001
|
|
- type: P@3(MIRACL)
|
|
value: 31.302000000000003
|
|
- type: P@5(MIRACL)
|
|
value: 23.384
|
|
- type: Recall@1(MIRACL)
|
|
value: 29.732999999999997
|
|
- type: Recall@10(MIRACL)
|
|
value: 66.532
|
|
- type: Recall@100(MIRACL)
|
|
value: 79.952
|
|
- type: Recall@1000(MIRACL)
|
|
value: 79.952
|
|
- type: Recall@20(MIRACL)
|
|
value: 73.75
|
|
- type: Recall@3(MIRACL)
|
|
value: 49.541000000000004
|
|
- type: Recall@5(MIRACL)
|
|
value: 57.389
|
|
- type: main_score
|
|
value: 56.111999999999995
|
|
- type: nAUC_MAP@1000_diff1(MIRACL)
|
|
value: 15.8510181843185
|
|
- type: nAUC_MAP@1000_max(MIRACL)
|
|
value: 27.452155305037095
|
|
- type: nAUC_MAP@1000_std(MIRACL)
|
|
value: 15.147015882448075
|
|
- type: nAUC_MAP@100_diff1(MIRACL)
|
|
value: 15.8510181843185
|
|
- type: nAUC_MAP@100_max(MIRACL)
|
|
value: 27.452155305037095
|
|
- type: nAUC_MAP@100_std(MIRACL)
|
|
value: 15.147015882448075
|
|
- type: nAUC_MAP@10_diff1(MIRACL)
|
|
value: 17.808742699385363
|
|
- type: nAUC_MAP@10_max(MIRACL)
|
|
value: 25.21217663908093
|
|
- type: nAUC_MAP@10_std(MIRACL)
|
|
value: 13.970995033749716
|
|
- type: nAUC_MAP@1_diff1(MIRACL)
|
|
value: 34.30066727981356
|
|
- type: nAUC_MAP@1_max(MIRACL)
|
|
value: 11.096793012814972
|
|
- type: nAUC_MAP@1_std(MIRACL)
|
|
value: 4.298644702770651
|
|
- type: nAUC_MAP@20_diff1(MIRACL)
|
|
value: 16.499957004860978
|
|
- type: nAUC_MAP@20_max(MIRACL)
|
|
value: 26.676987318433714
|
|
- type: nAUC_MAP@20_std(MIRACL)
|
|
value: 15.166175199040485
|
|
- type: nAUC_MAP@3_diff1(MIRACL)
|
|
value: 23.797870452650084
|
|
- type: nAUC_MAP@3_max(MIRACL)
|
|
value: 18.20460307122738
|
|
- type: nAUC_MAP@3_std(MIRACL)
|
|
value: 8.985118628338126
|
|
- type: nAUC_MAP@5_diff1(MIRACL)
|
|
value: 20.549029352694866
|
|
- type: nAUC_MAP@5_max(MIRACL)
|
|
value: 21.528805328834324
|
|
- type: nAUC_MAP@5_std(MIRACL)
|
|
value: 11.131951589460492
|
|
- type: nAUC_NDCG@1000_diff1(MIRACL)
|
|
value: 5.973372149854828
|
|
- type: nAUC_NDCG@1000_max(MIRACL)
|
|
value: 36.70565868748619
|
|
- type: nAUC_NDCG@1000_std(MIRACL)
|
|
value: 19.551007976769245
|
|
- type: nAUC_NDCG@100_diff1(MIRACL)
|
|
value: 5.973372149854828
|
|
- type: nAUC_NDCG@100_max(MIRACL)
|
|
value: 36.70565868748619
|
|
- type: nAUC_NDCG@100_std(MIRACL)
|
|
value: 19.551007976769245
|
|
- type: nAUC_NDCG@10_diff1(MIRACL)
|
|
value: 10.894100451667919
|
|
- type: nAUC_NDCG@10_max(MIRACL)
|
|
value: 31.735109695399416
|
|
- type: nAUC_NDCG@10_std(MIRACL)
|
|
value: 17.674556265190706
|
|
- type: nAUC_NDCG@1_diff1(MIRACL)
|
|
value: 22.04892839322977
|
|
- type: nAUC_NDCG@1_max(MIRACL)
|
|
value: 32.51034181981298
|
|
- type: nAUC_NDCG@1_std(MIRACL)
|
|
value: 14.343760356007765
|
|
- type: nAUC_NDCG@20_diff1(MIRACL)
|
|
value: 8.074119776676103
|
|
- type: nAUC_NDCG@20_max(MIRACL)
|
|
value: 34.52221220694718
|
|
- type: nAUC_NDCG@20_std(MIRACL)
|
|
value: 19.94006423667
|
|
- type: nAUC_NDCG@3_diff1(MIRACL)
|
|
value: 16.284195830367825
|
|
- type: nAUC_NDCG@3_max(MIRACL)
|
|
value: 26.521965826220352
|
|
- type: nAUC_NDCG@3_std(MIRACL)
|
|
value: 13.850033289666094
|
|
- type: nAUC_NDCG@5_diff1(MIRACL)
|
|
value: 14.362693198633952
|
|
- type: nAUC_NDCG@5_max(MIRACL)
|
|
value: 27.781809390068872
|
|
- type: nAUC_NDCG@5_std(MIRACL)
|
|
value: 14.879808284537981
|
|
- type: nAUC_P@1000_diff1(MIRACL)
|
|
value: -27.606682296231373
|
|
- type: nAUC_P@1000_max(MIRACL)
|
|
value: 33.03084251491326
|
|
- type: nAUC_P@1000_std(MIRACL)
|
|
value: 15.674013757663898
|
|
- type: nAUC_P@100_diff1(MIRACL)
|
|
value: -27.606682296231327
|
|
- type: nAUC_P@100_max(MIRACL)
|
|
value: 33.03084251491332
|
|
- type: nAUC_P@100_std(MIRACL)
|
|
value: 15.674013757663937
|
|
- type: nAUC_P@10_diff1(MIRACL)
|
|
value: -23.575685602922174
|
|
- type: nAUC_P@10_max(MIRACL)
|
|
value: 36.72548498655645
|
|
- type: nAUC_P@10_std(MIRACL)
|
|
value: 21.317694028285104
|
|
- type: nAUC_P@1_diff1(MIRACL)
|
|
value: 22.04892839322977
|
|
- type: nAUC_P@1_max(MIRACL)
|
|
value: 32.51034181981298
|
|
- type: nAUC_P@1_std(MIRACL)
|
|
value: 14.343760356007765
|
|
- type: nAUC_P@20_diff1(MIRACL)
|
|
value: -26.064734965649322
|
|
- type: nAUC_P@20_max(MIRACL)
|
|
value: 34.10936682680113
|
|
- type: nAUC_P@20_std(MIRACL)
|
|
value: 20.31615496254574
|
|
- type: nAUC_P@3_diff1(MIRACL)
|
|
value: -10.903444655544746
|
|
- type: nAUC_P@3_max(MIRACL)
|
|
value: 34.33585029049373
|
|
- type: nAUC_P@3_std(MIRACL)
|
|
value: 18.620142249622834
|
|
- type: nAUC_P@5_diff1(MIRACL)
|
|
value: -18.454884144221385
|
|
- type: nAUC_P@5_max(MIRACL)
|
|
value: 35.620428961110036
|
|
- type: nAUC_P@5_std(MIRACL)
|
|
value: 20.265460635926893
|
|
- type: nAUC_Recall@1000_diff1(MIRACL)
|
|
value: -28.25716669219796
|
|
- type: nAUC_Recall@1000_max(MIRACL)
|
|
value: 59.88673755432144
|
|
- type: nAUC_Recall@1000_std(MIRACL)
|
|
value: 29.916576785101622
|
|
- type: nAUC_Recall@100_diff1(MIRACL)
|
|
value: -28.25716669219796
|
|
- type: nAUC_Recall@100_max(MIRACL)
|
|
value: 59.88673755432144
|
|
- type: nAUC_Recall@100_std(MIRACL)
|
|
value: 29.916576785101622
|
|
- type: nAUC_Recall@10_diff1(MIRACL)
|
|
value: -2.5731369116803466
|
|
- type: nAUC_Recall@10_max(MIRACL)
|
|
value: 34.37108435281944
|
|
- type: nAUC_Recall@10_std(MIRACL)
|
|
value: 20.744457001608925
|
|
- type: nAUC_Recall@1_diff1(MIRACL)
|
|
value: 34.30066727981356
|
|
- type: nAUC_Recall@1_max(MIRACL)
|
|
value: 11.096793012814972
|
|
- type: nAUC_Recall@1_std(MIRACL)
|
|
value: 4.298644702770651
|
|
- type: nAUC_Recall@20_diff1(MIRACL)
|
|
value: -13.667980220614172
|
|
- type: nAUC_Recall@20_max(MIRACL)
|
|
value: 44.947659106700044
|
|
- type: nAUC_Recall@20_std(MIRACL)
|
|
value: 29.413435369376923
|
|
- type: nAUC_Recall@3_diff1(MIRACL)
|
|
value: 15.838199908854786
|
|
- type: nAUC_Recall@3_max(MIRACL)
|
|
value: 17.368565662731196
|
|
- type: nAUC_Recall@3_std(MIRACL)
|
|
value: 10.538072940876807
|
|
- type: nAUC_Recall@5_diff1(MIRACL)
|
|
value: 8.199967584892176
|
|
- type: nAUC_Recall@5_max(MIRACL)
|
|
value: 23.500985460573578
|
|
- type: nAUC_Recall@5_std(MIRACL)
|
|
value: 13.477424183539433
|
|
task:
|
|
type: Reranking
|
|
- dataset:
|
|
config: ru
|
|
name: MTEB MIRACLRetrieval (ru)
|
|
revision: main
|
|
split: dev
|
|
type: miracl/mmteb-miracl
|
|
metrics:
|
|
- type: main_score
|
|
value: 52.211
|
|
- type: map_at_1
|
|
value: 23.238
|
|
- type: map_at_10
|
|
value: 41.559000000000005
|
|
- type: map_at_100
|
|
value: 44.757999999999996
|
|
- type: map_at_1000
|
|
value: 44.861000000000004
|
|
- type: map_at_20
|
|
value: 43.461
|
|
- type: map_at_3
|
|
value: 34.593
|
|
- type: map_at_5
|
|
value: 38.056
|
|
- type: mrr_at_1
|
|
value: 47.04472843450479
|
|
- type: mrr_at_10
|
|
value: 59.587485420153286
|
|
- type: mrr_at_100
|
|
value: 60.17662556783717
|
|
- type: mrr_at_1000
|
|
value: 60.1850174860852
|
|
- type: mrr_at_20
|
|
value: 60.003979383733544
|
|
- type: mrr_at_3
|
|
value: 56.62939297124608
|
|
- type: mrr_at_5
|
|
value: 58.33067092651768
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 26.665139374258256
|
|
- type: nauc_map_at_1000_max
|
|
value: 20.20801190375824
|
|
- type: nauc_map_at_1000_std
|
|
value: 3.35434510540552
|
|
- type: nauc_map_at_100_diff1
|
|
value: 26.691816652639787
|
|
- type: nauc_map_at_100_max
|
|
value: 20.193510183457917
|
|
- type: nauc_map_at_100_std
|
|
value: 3.371679544337864
|
|
- type: nauc_map_at_10_diff1
|
|
value: 27.24904607990151
|
|
- type: nauc_map_at_10_max
|
|
value: 18.26589731339405
|
|
- type: nauc_map_at_10_std
|
|
value: 1.0177924180874538
|
|
- type: nauc_map_at_1_diff1
|
|
value: 34.53595808193455
|
|
- type: nauc_map_at_1_max
|
|
value: 10.970155439499656
|
|
- type: nauc_map_at_1_std
|
|
value: -3.8626873246816373
|
|
- type: nauc_map_at_20_diff1
|
|
value: 26.8513788979128
|
|
- type: nauc_map_at_20_max
|
|
value: 19.367475736662428
|
|
- type: nauc_map_at_20_std
|
|
value: 2.2475091146613564
|
|
- type: nauc_map_at_3_diff1
|
|
value: 28.911815196615866
|
|
- type: nauc_map_at_3_max
|
|
value: 15.474121149651292
|
|
- type: nauc_map_at_3_std
|
|
value: -1.0664535264565158
|
|
- type: nauc_map_at_5_diff1
|
|
value: 27.772031743222787
|
|
- type: nauc_map_at_5_max
|
|
value: 16.241638808384145
|
|
- type: nauc_map_at_5_std
|
|
value: -0.6044307972013538
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 26.66563442138901
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 27.74734004586503
|
|
- type: nauc_mrr_at_1000_std
|
|
value: 10.663042801330587
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 26.66809693875436
|
|
- type: nauc_mrr_at_100_max
|
|
value: 27.7565667281779
|
|
- type: nauc_mrr_at_100_std
|
|
value: 10.671838040923266
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 26.587658592417736
|
|
- type: nauc_mrr_at_10_max
|
|
value: 27.872712998242328
|
|
- type: nauc_mrr_at_10_std
|
|
value: 10.979716151856918
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 29.30751401472168
|
|
- type: nauc_mrr_at_1_max
|
|
value: 24.98212676568516
|
|
- type: nauc_mrr_at_1_std
|
|
value: 6.094206809391165
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 26.52396413399926
|
|
- type: nauc_mrr_at_20_max
|
|
value: 27.720568784204847
|
|
- type: nauc_mrr_at_20_std
|
|
value: 10.749903126459412
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 26.993782403961802
|
|
- type: nauc_mrr_at_3_max
|
|
value: 27.810128603605342
|
|
- type: nauc_mrr_at_3_std
|
|
value: 10.526250026174825
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 26.491056284663404
|
|
- type: nauc_mrr_at_5_max
|
|
value: 27.938292238745838
|
|
- type: nauc_mrr_at_5_std
|
|
value: 10.620036152236098
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 24.743263734342236
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 25.632023742967196
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: 9.54979482991325
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 24.884477288371073
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 25.856099754401797
|
|
- type: nauc_ndcg_at_100_std
|
|
value: 10.275002448873611
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 25.813663674330005
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 21.4632558325771
|
|
- type: nauc_ndcg_at_10_std
|
|
value: 4.793772488457711
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 29.30751401472168
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 24.98212676568516
|
|
- type: nauc_ndcg_at_1_std
|
|
value: 6.094206809391165
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 24.96712085611002
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 23.176681160212546
|
|
- type: nauc_ndcg_at_20_std
|
|
value: 6.936886476037671
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 25.475637018641205
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 22.040672063815855
|
|
- type: nauc_ndcg_at_3_std
|
|
value: 5.327531594448605
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 25.70702625003538
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 20.273499330943313
|
|
- type: nauc_ndcg_at_5_std
|
|
value: 3.733783938564952
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -14.918023025551047
|
|
- type: nauc_precision_at_1000_max
|
|
value: 18.668936317187704
|
|
- type: nauc_precision_at_1000_std
|
|
value: 19.15643973163778
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -12.902497092152561
|
|
- type: nauc_precision_at_100_max
|
|
value: 22.117700522212857
|
|
- type: nauc_precision_at_100_std
|
|
value: 23.367379142816734
|
|
- type: nauc_precision_at_10_diff1
|
|
value: -3.319884895143968
|
|
- type: nauc_precision_at_10_max
|
|
value: 25.207453700919412
|
|
- type: nauc_precision_at_10_std
|
|
value: 16.768944029523773
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 29.30751401472168
|
|
- type: nauc_precision_at_1_max
|
|
value: 24.98212676568516
|
|
- type: nauc_precision_at_1_std
|
|
value: 6.094206809391165
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -8.101925051455304
|
|
- type: nauc_precision_at_20_max
|
|
value: 23.93155685736234
|
|
- type: nauc_precision_at_20_std
|
|
value: 19.599852197885983
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 8.604157546918138
|
|
- type: nauc_precision_at_3_max
|
|
value: 26.8274074367336
|
|
- type: nauc_precision_at_3_std
|
|
value: 13.210078569814973
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 2.0240126571446004
|
|
- type: nauc_precision_at_5_max
|
|
value: 25.068271323836683
|
|
- type: nauc_precision_at_5_std
|
|
value: 13.423044252359
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 2.5057442905176264
|
|
- type: nauc_recall_at_1000_max
|
|
value: 57.765040045333485
|
|
- type: nauc_recall_at_1000_std
|
|
value: 75.40225417846978
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 13.982399962667946
|
|
- type: nauc_recall_at_100_max
|
|
value: 36.06499090419987
|
|
- type: nauc_recall_at_100_std
|
|
value: 38.55877836909554
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 19.09907433139298
|
|
- type: nauc_recall_at_10_max
|
|
value: 14.320755651797818
|
|
- type: nauc_recall_at_10_std
|
|
value: 3.68835109545608
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 34.53595808193455
|
|
- type: nauc_recall_at_1_max
|
|
value: 10.970155439499656
|
|
- type: nauc_recall_at_1_std
|
|
value: -3.8626873246816373
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 15.80854510984775
|
|
- type: nauc_recall_at_20_max
|
|
value: 17.20627614536354
|
|
- type: nauc_recall_at_20_std
|
|
value: 9.028188051323042
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 23.88853757885772
|
|
- type: nauc_recall_at_3_max
|
|
value: 13.29954353582913
|
|
- type: nauc_recall_at_3_std
|
|
value: -0.42190904806759966
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 20.720312115028822
|
|
- type: nauc_recall_at_5_max
|
|
value: 12.324541527710025
|
|
- type: nauc_recall_at_5_std
|
|
value: -0.19420222400103399
|
|
- type: ndcg_at_1
|
|
value: 47.044999999999995
|
|
- type: ndcg_at_10
|
|
value: 52.211
|
|
- type: ndcg_at_100
|
|
value: 60.777
|
|
- type: ndcg_at_1000
|
|
value: 61.951
|
|
- type: ndcg_at_20
|
|
value: 56.215
|
|
- type: ndcg_at_3
|
|
value: 45.871
|
|
- type: ndcg_at_5
|
|
value: 47.643
|
|
- type: precision_at_1
|
|
value: 47.044999999999995
|
|
- type: precision_at_10
|
|
value: 16.062
|
|
- type: precision_at_100
|
|
value: 2.563
|
|
- type: precision_at_1000
|
|
value: 0.27899999999999997
|
|
- type: precision_at_20
|
|
value: 9.9
|
|
- type: precision_at_3
|
|
value: 31.575999999999997
|
|
- type: precision_at_5
|
|
value: 24.153
|
|
- type: recall_at_1
|
|
value: 23.238
|
|
- type: recall_at_10
|
|
value: 63.479
|
|
- type: recall_at_100
|
|
value: 91.51899999999999
|
|
- type: recall_at_1000
|
|
value: 97.906
|
|
- type: recall_at_20
|
|
value: 74.705
|
|
- type: recall_at_3
|
|
value: 42.082
|
|
- type: recall_at_5
|
|
value: 50.708
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: ru
|
|
name: MTEB MassiveIntentClassification (ru)
|
|
revision: 4672e20407010da34463acc759c162ca9734bca6
|
|
split: test
|
|
type: mteb/amazon_massive_intent
|
|
metrics:
|
|
- type: accuracy
|
|
value: 68.61466039004706
|
|
- type: f1
|
|
value: 63.790707574282045
|
|
- type: f1_weighted
|
|
value: 67.28456899088164
|
|
- type: main_score
|
|
value: 68.61466039004706
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: ru
|
|
name: MTEB MassiveScenarioClassification (ru)
|
|
revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
|
|
split: test
|
|
type: mteb/amazon_massive_scenario
|
|
metrics:
|
|
- type: accuracy
|
|
value: 77.97579018157364
|
|
- type: f1
|
|
value: 76.31497051309336
|
|
- type: f1_weighted
|
|
value: 77.54198422119202
|
|
- type: main_score
|
|
value: 77.97579018157364
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB RUParaPhraserSTS
|
|
revision: 43265056790b8f7c59e0139acb4be0a8dad2c8f4
|
|
split: test
|
|
type: merionum/ru_paraphraser
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 62.072853635744465
|
|
- type: cosine_spearman
|
|
value: 68.32627155640247
|
|
- type: euclidean_pearson
|
|
value: 65.56072460948485
|
|
- type: euclidean_spearman
|
|
value: 68.32632364995054
|
|
- type: main_score
|
|
value: 68.32627155640247
|
|
- type: manhattan_pearson
|
|
value: 65.54799770948776
|
|
- type: manhattan_spearman
|
|
value: 68.2428132570697
|
|
- type: pearson
|
|
value: 62.072853635744465
|
|
- type: spearman
|
|
value: 68.32627155640247
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB RiaNewsRetrieval
|
|
revision: 82374b0bbacda6114f39ff9c5b925fa1512ca5d7
|
|
split: test
|
|
type: ai-forever/ria-news-retrieval
|
|
metrics:
|
|
- type: main_score
|
|
value: 79.42399999999999
|
|
- type: map_at_1
|
|
value: 67.42
|
|
- type: map_at_10
|
|
value: 75.81700000000001
|
|
- type: map_at_100
|
|
value: 76.103
|
|
- type: map_at_1000
|
|
value: 76.11099999999999
|
|
- type: map_at_20
|
|
value: 76.011
|
|
- type: map_at_3
|
|
value: 74.38
|
|
- type: map_at_5
|
|
value: 75.31400000000001
|
|
- type: mrr_at_1
|
|
value: 67.42
|
|
- type: mrr_at_10
|
|
value: 75.81702380952322
|
|
- type: mrr_at_100
|
|
value: 76.10294206257022
|
|
- type: mrr_at_1000
|
|
value: 76.11127333184083
|
|
- type: mrr_at_20
|
|
value: 76.01092756817413
|
|
- type: mrr_at_3
|
|
value: 74.37999999999947
|
|
- type: mrr_at_5
|
|
value: 75.31449999999931
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 74.47312749692254
|
|
- type: nauc_map_at_1000_max
|
|
value: 24.255650636762592
|
|
- type: nauc_map_at_1000_std
|
|
value: -13.538045103707466
|
|
- type: nauc_map_at_100_diff1
|
|
value: 74.46935527123232
|
|
- type: nauc_map_at_100_max
|
|
value: 24.260637479032273
|
|
- type: nauc_map_at_100_std
|
|
value: -13.526893488105108
|
|
- type: nauc_map_at_10_diff1
|
|
value: 74.37904649319015
|
|
- type: nauc_map_at_10_max
|
|
value: 24.25477514829031
|
|
- type: nauc_map_at_10_std
|
|
value: -13.673101053529166
|
|
- type: nauc_map_at_1_diff1
|
|
value: 77.41742450291483
|
|
- type: nauc_map_at_1_max
|
|
value: 21.561634939014
|
|
- type: nauc_map_at_1_std
|
|
value: -15.302925641163046
|
|
- type: nauc_map_at_20_diff1
|
|
value: 74.44339113303336
|
|
- type: nauc_map_at_20_max
|
|
value: 24.281346979231508
|
|
- type: nauc_map_at_20_std
|
|
value: -13.533874833150467
|
|
- type: nauc_map_at_3_diff1
|
|
value: 74.31017752460161
|
|
- type: nauc_map_at_3_max
|
|
value: 24.209272036097506
|
|
- type: nauc_map_at_3_std
|
|
value: -14.053104049162751
|
|
- type: nauc_map_at_5_diff1
|
|
value: 74.42859541067173
|
|
- type: nauc_map_at_5_max
|
|
value: 24.16570861589971
|
|
- type: nauc_map_at_5_std
|
|
value: -13.948432311463257
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 74.47312785315074
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 24.255652429274488
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -13.538043692357599
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 74.46935527123232
|
|
- type: nauc_mrr_at_100_max
|
|
value: 24.260637479032273
|
|
- type: nauc_mrr_at_100_std
|
|
value: -13.526893488105108
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 74.37904649319015
|
|
- type: nauc_mrr_at_10_max
|
|
value: 24.25477514829031
|
|
- type: nauc_mrr_at_10_std
|
|
value: -13.673101053529166
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 77.41742450291483
|
|
- type: nauc_mrr_at_1_max
|
|
value: 21.561634939014
|
|
- type: nauc_mrr_at_1_std
|
|
value: -15.302925641163046
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 74.44339113303336
|
|
- type: nauc_mrr_at_20_max
|
|
value: 24.281346979231508
|
|
- type: nauc_mrr_at_20_std
|
|
value: -13.533874833150467
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 74.31017752460161
|
|
- type: nauc_mrr_at_3_max
|
|
value: 24.209272036097506
|
|
- type: nauc_mrr_at_3_std
|
|
value: -14.053104049162751
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 74.42859541067173
|
|
- type: nauc_mrr_at_5_max
|
|
value: 24.16570861589971
|
|
- type: nauc_mrr_at_5_std
|
|
value: -13.948432311463257
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 73.67049349073889
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 25.36219767677513
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -12.018149673769434
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 73.52540106541404
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 25.54104779422804
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -11.596858470683141
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 73.13668875552696
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 25.555285618887662
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -12.31485256997023
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 77.41742450291483
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 21.561634939014
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -15.302925641163046
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 73.35771732216482
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 25.73112191366883
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -11.69854261340669
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 73.20274751289709
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 25.285529084214925
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -13.37770120862227
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 73.33594229336342
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 25.281830078361644
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -13.088615162069974
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: 55.90120106013352
|
|
- type: nauc_precision_at_1000_max
|
|
value: 55.70083105705886
|
|
- type: nauc_precision_at_1000_std
|
|
value: 36.2217350708384
|
|
- type: nauc_precision_at_100_diff1
|
|
value: 59.2870776629234
|
|
- type: nauc_precision_at_100_max
|
|
value: 47.133189559008834
|
|
- type: nauc_precision_at_100_std
|
|
value: 28.301920571571802
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 65.12757705051081
|
|
- type: nauc_precision_at_10_max
|
|
value: 34.0578425463014
|
|
- type: nauc_precision_at_10_std
|
|
value: -2.7826038995063618
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 77.41742450291483
|
|
- type: nauc_precision_at_1_max
|
|
value: 21.561634939014
|
|
- type: nauc_precision_at_1_std
|
|
value: -15.302925641163046
|
|
- type: nauc_precision_at_20_diff1
|
|
value: 64.13592064044578
|
|
- type: nauc_precision_at_20_max
|
|
value: 39.3269437952694
|
|
- type: nauc_precision_at_20_std
|
|
value: 7.181669511985859
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 68.88283614651107
|
|
- type: nauc_precision_at_3_max
|
|
value: 29.546078723110387
|
|
- type: nauc_precision_at_3_std
|
|
value: -10.635148066667597
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 68.11610612745827
|
|
- type: nauc_precision_at_5_max
|
|
value: 30.708733892411683
|
|
- type: nauc_precision_at_5_std
|
|
value: -8.722606142068399
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: 55.90120106013372
|
|
- type: nauc_recall_at_1000_max
|
|
value: 55.70083105705975
|
|
- type: nauc_recall_at_1000_std
|
|
value: 36.22173507083937
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 59.287077662923856
|
|
- type: nauc_recall_at_100_max
|
|
value: 47.1331895590096
|
|
- type: nauc_recall_at_100_std
|
|
value: 28.30192057157174
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 65.1275770505108
|
|
- type: nauc_recall_at_10_max
|
|
value: 34.057842546301245
|
|
- type: nauc_recall_at_10_std
|
|
value: -2.7826038995065376
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 77.41742450291483
|
|
- type: nauc_recall_at_1_max
|
|
value: 21.561634939014
|
|
- type: nauc_recall_at_1_std
|
|
value: -15.302925641163046
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 64.13592064044556
|
|
- type: nauc_recall_at_20_max
|
|
value: 39.32694379526965
|
|
- type: nauc_recall_at_20_std
|
|
value: 7.181669511986287
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 68.88283614651114
|
|
- type: nauc_recall_at_3_max
|
|
value: 29.54607872311032
|
|
- type: nauc_recall_at_3_std
|
|
value: -10.635148066667742
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 68.11610612745811
|
|
- type: nauc_recall_at_5_max
|
|
value: 30.70873389241151
|
|
- type: nauc_recall_at_5_std
|
|
value: -8.722606142068207
|
|
- type: ndcg_at_1
|
|
value: 67.42
|
|
- type: ndcg_at_10
|
|
value: 79.42399999999999
|
|
- type: ndcg_at_100
|
|
value: 80.754
|
|
- type: ndcg_at_1000
|
|
value: 80.979
|
|
- type: ndcg_at_20
|
|
value: 80.118
|
|
- type: ndcg_at_3
|
|
value: 76.543
|
|
- type: ndcg_at_5
|
|
value: 78.215
|
|
- type: precision_at_1
|
|
value: 67.42
|
|
- type: precision_at_10
|
|
value: 9.052
|
|
- type: precision_at_100
|
|
value: 0.966
|
|
- type: precision_at_1000
|
|
value: 0.098
|
|
- type: precision_at_20
|
|
value: 4.662
|
|
- type: precision_at_3
|
|
value: 27.589999999999996
|
|
- type: precision_at_5
|
|
value: 17.36
|
|
- type: recall_at_1
|
|
value: 67.42
|
|
- type: recall_at_10
|
|
value: 90.52
|
|
- type: recall_at_100
|
|
value: 96.61
|
|
- type: recall_at_1000
|
|
value: 98.39
|
|
- type: recall_at_20
|
|
value: 93.24
|
|
- type: recall_at_3
|
|
value: 82.77
|
|
- type: recall_at_5
|
|
value: 86.8
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB RuBQReranking
|
|
revision: 2e96b8f098fa4b0950fc58eacadeb31c0d0c7fa2
|
|
split: test
|
|
type: ai-forever/rubq-reranking
|
|
metrics:
|
|
- type: main_score
|
|
value: 68.48180892753541
|
|
- type: map
|
|
value: 68.48180892753541
|
|
- type: mrr
|
|
value: 73.69372550223615
|
|
- type: nAUC_map_diff1
|
|
value: 37.93778560797301
|
|
- type: nAUC_map_max
|
|
value: 10.858022431340633
|
|
- type: nAUC_map_std
|
|
value: 6.446466714820493
|
|
- type: nAUC_mrr_diff1
|
|
value: 39.83698029227208
|
|
- type: nAUC_mrr_max
|
|
value: 14.378309445768284
|
|
- type: nAUC_mrr_std
|
|
value: 10.579567761464919
|
|
task:
|
|
type: Reranking
|
|
- dataset:
|
|
config: default
|
|
name: MTEB RuBQRetrieval
|
|
revision: e19b6ffa60b3bc248e0b41f4cc37c26a55c2a67b
|
|
split: test
|
|
type: ai-forever/rubq-retrieval
|
|
metrics:
|
|
- type: main_score
|
|
value: 66.77
|
|
- type: map_at_1
|
|
value: 36.525
|
|
- type: map_at_10
|
|
value: 58.021
|
|
- type: map_at_100
|
|
value: 59.016000000000005
|
|
- type: map_at_1000
|
|
value: 59.041999999999994
|
|
- type: map_at_20
|
|
value: 58.709
|
|
- type: map_at_3
|
|
value: 51.808
|
|
- type: map_at_5
|
|
value: 55.706999999999994
|
|
- type: mrr_at_1
|
|
value: 52.95508274231678
|
|
- type: mrr_at_10
|
|
value: 66.10029926076034
|
|
- type: mrr_at_100
|
|
value: 66.46489903689454
|
|
- type: mrr_at_1000
|
|
value: 66.47135430048212
|
|
- type: mrr_at_20
|
|
value: 66.36282360130573
|
|
- type: mrr_at_3
|
|
value: 63.347123719464236
|
|
- type: mrr_at_5
|
|
value: 65.20291568163925
|
|
- type: nauc_map_at_1000_diff1
|
|
value: 36.39353112777031
|
|
- type: nauc_map_at_1000_max
|
|
value: 14.511234479555156
|
|
- type: nauc_map_at_1000_std
|
|
value: -12.003784393055856
|
|
- type: nauc_map_at_100_diff1
|
|
value: 36.396297354858326
|
|
- type: nauc_map_at_100_max
|
|
value: 14.532932252459755
|
|
- type: nauc_map_at_100_std
|
|
value: -11.9933713072409
|
|
- type: nauc_map_at_10_diff1
|
|
value: 36.19731963995984
|
|
- type: nauc_map_at_10_max
|
|
value: 14.331593327284844
|
|
- type: nauc_map_at_10_std
|
|
value: -12.607001882190588
|
|
- type: nauc_map_at_1_diff1
|
|
value: 39.04224394212489
|
|
- type: nauc_map_at_1_max
|
|
value: 9.44079807509392
|
|
- type: nauc_map_at_1_std
|
|
value: -8.725551038382205
|
|
- type: nauc_map_at_20_diff1
|
|
value: 36.27250811060138
|
|
- type: nauc_map_at_20_max
|
|
value: 14.521970331255876
|
|
- type: nauc_map_at_20_std
|
|
value: -12.033391150828098
|
|
- type: nauc_map_at_3_diff1
|
|
value: 35.966460233965485
|
|
- type: nauc_map_at_3_max
|
|
value: 11.62955834976298
|
|
- type: nauc_map_at_3_std
|
|
value: -13.649024048480133
|
|
- type: nauc_map_at_5_diff1
|
|
value: 36.131815002934644
|
|
- type: nauc_map_at_5_max
|
|
value: 13.157509275481777
|
|
- type: nauc_map_at_5_std
|
|
value: -13.36839170298778
|
|
- type: nauc_mrr_at_1000_diff1
|
|
value: 40.191647456610056
|
|
- type: nauc_mrr_at_1000_max
|
|
value: 16.63142892913043
|
|
- type: nauc_mrr_at_1000_std
|
|
value: -12.671951113868769
|
|
- type: nauc_mrr_at_100_diff1
|
|
value: 40.18726742271696
|
|
- type: nauc_mrr_at_100_max
|
|
value: 16.638314382103207
|
|
- type: nauc_mrr_at_100_std
|
|
value: -12.664912420744438
|
|
- type: nauc_mrr_at_10_diff1
|
|
value: 40.028293277796855
|
|
- type: nauc_mrr_at_10_max
|
|
value: 16.841638035795718
|
|
- type: nauc_mrr_at_10_std
|
|
value: -12.781785759758687
|
|
- type: nauc_mrr_at_1_diff1
|
|
value: 42.26303997344821
|
|
- type: nauc_mrr_at_1_max
|
|
value: 14.211014905785252
|
|
- type: nauc_mrr_at_1_std
|
|
value: -11.030701637062437
|
|
- type: nauc_mrr_at_20_diff1
|
|
value: 40.12680433695074
|
|
- type: nauc_mrr_at_20_max
|
|
value: 16.75915749592042
|
|
- type: nauc_mrr_at_20_std
|
|
value: -12.613807048523782
|
|
- type: nauc_mrr_at_3_diff1
|
|
value: 40.32434278687767
|
|
- type: nauc_mrr_at_3_max
|
|
value: 15.811615950737387
|
|
- type: nauc_mrr_at_3_std
|
|
value: -13.957860180387636
|
|
- type: nauc_mrr_at_5_diff1
|
|
value: 40.09422159913817
|
|
- type: nauc_mrr_at_5_max
|
|
value: 16.64090259238879
|
|
- type: nauc_mrr_at_5_std
|
|
value: -13.230746065794726
|
|
- type: nauc_ndcg_at_1000_diff1
|
|
value: 36.67352791454268
|
|
- type: nauc_ndcg_at_1000_max
|
|
value: 16.749915190801016
|
|
- type: nauc_ndcg_at_1000_std
|
|
value: -11.008545008175378
|
|
- type: nauc_ndcg_at_100_diff1
|
|
value: 36.58072887287039
|
|
- type: nauc_ndcg_at_100_max
|
|
value: 17.22374718832945
|
|
- type: nauc_ndcg_at_100_std
|
|
value: -10.559637745205016
|
|
- type: nauc_ndcg_at_10_diff1
|
|
value: 35.786024269753334
|
|
- type: nauc_ndcg_at_10_max
|
|
value: 17.217091860749864
|
|
- type: nauc_ndcg_at_10_std
|
|
value: -12.505927857541066
|
|
- type: nauc_ndcg_at_1_diff1
|
|
value: 42.41055520049291
|
|
- type: nauc_ndcg_at_1_max
|
|
value: 14.001922648893919
|
|
- type: nauc_ndcg_at_1_std
|
|
value: -11.224085018036103
|
|
- type: nauc_ndcg_at_20_diff1
|
|
value: 35.9577978619838
|
|
- type: nauc_ndcg_at_20_max
|
|
value: 17.612142353807204
|
|
- type: nauc_ndcg_at_20_std
|
|
value: -10.715656533623179
|
|
- type: nauc_ndcg_at_3_diff1
|
|
value: 35.92331458170165
|
|
- type: nauc_ndcg_at_3_max
|
|
value: 12.972908846104833
|
|
- type: nauc_ndcg_at_3_std
|
|
value: -14.90499944816046
|
|
- type: nauc_ndcg_at_5_diff1
|
|
value: 35.87509174776851
|
|
- type: nauc_ndcg_at_5_max
|
|
value: 15.016606655112842
|
|
- type: nauc_ndcg_at_5_std
|
|
value: -14.252766370474959
|
|
- type: nauc_precision_at_1000_diff1
|
|
value: -7.854237065573715
|
|
- type: nauc_precision_at_1000_max
|
|
value: 7.340193640831781
|
|
- type: nauc_precision_at_1000_std
|
|
value: 5.270139452495764
|
|
- type: nauc_precision_at_100_diff1
|
|
value: -5.433762342336105
|
|
- type: nauc_precision_at_100_max
|
|
value: 10.323131724715576
|
|
- type: nauc_precision_at_100_std
|
|
value: 6.065361232063088
|
|
- type: nauc_precision_at_10_diff1
|
|
value: 1.6163013309854788
|
|
- type: nauc_precision_at_10_max
|
|
value: 13.853149437703955
|
|
- type: nauc_precision_at_10_std
|
|
value: -0.4630873244645538
|
|
- type: nauc_precision_at_1_diff1
|
|
value: 42.41055520049291
|
|
- type: nauc_precision_at_1_max
|
|
value: 14.001922648893919
|
|
- type: nauc_precision_at_1_std
|
|
value: -11.224085018036103
|
|
- type: nauc_precision_at_20_diff1
|
|
value: -2.406608082278331
|
|
- type: nauc_precision_at_20_max
|
|
value: 12.672408320017443
|
|
- type: nauc_precision_at_20_std
|
|
value: 4.420612595577876
|
|
- type: nauc_precision_at_3_diff1
|
|
value: 15.724555799730243
|
|
- type: nauc_precision_at_3_max
|
|
value: 12.818558415088615
|
|
- type: nauc_precision_at_3_std
|
|
value: -11.49979730611224
|
|
- type: nauc_precision_at_5_diff1
|
|
value: 8.485573750280292
|
|
- type: nauc_precision_at_5_max
|
|
value: 13.304773839372094
|
|
- type: nauc_precision_at_5_std
|
|
value: -6.633911950881821
|
|
- type: nauc_recall_at_1000_diff1
|
|
value: -7.902591492154048
|
|
- type: nauc_recall_at_1000_max
|
|
value: 54.202835032879946
|
|
- type: nauc_recall_at_1000_std
|
|
value: 68.22401286555711
|
|
- type: nauc_recall_at_100_diff1
|
|
value: 14.88281690495126
|
|
- type: nauc_recall_at_100_max
|
|
value: 41.9305338281276
|
|
- type: nauc_recall_at_100_std
|
|
value: 30.260295038603324
|
|
- type: nauc_recall_at_10_diff1
|
|
value: 23.09613458762812
|
|
- type: nauc_recall_at_10_max
|
|
value: 24.921985669652386
|
|
- type: nauc_recall_at_10_std
|
|
value: -9.990910822464661
|
|
- type: nauc_recall_at_1_diff1
|
|
value: 39.04224394212489
|
|
- type: nauc_recall_at_1_max
|
|
value: 9.44079807509392
|
|
- type: nauc_recall_at_1_std
|
|
value: -8.725551038382205
|
|
- type: nauc_recall_at_20_diff1
|
|
value: 19.41298369752395
|
|
- type: nauc_recall_at_20_max
|
|
value: 31.91169321346991
|
|
- type: nauc_recall_at_20_std
|
|
value: 4.514353181881159
|
|
- type: nauc_recall_at_3_diff1
|
|
value: 29.514018426239197
|
|
- type: nauc_recall_at_3_max
|
|
value: 10.600179069626673
|
|
- type: nauc_recall_at_3_std
|
|
value: -17.02685998662361
|
|
- type: nauc_recall_at_5_diff1
|
|
value: 26.66966838912029
|
|
- type: nauc_recall_at_5_max
|
|
value: 15.359436829533934
|
|
- type: nauc_recall_at_5_std
|
|
value: -15.87666175175801
|
|
- type: ndcg_at_1
|
|
value: 52.896
|
|
- type: ndcg_at_10
|
|
value: 66.77
|
|
- type: ndcg_at_100
|
|
value: 69.98100000000001
|
|
- type: ndcg_at_1000
|
|
value: 70.408
|
|
- type: ndcg_at_20
|
|
value: 68.53200000000001
|
|
- type: ndcg_at_3
|
|
value: 58.074999999999996
|
|
- type: ndcg_at_5
|
|
value: 62.841
|
|
- type: precision_at_1
|
|
value: 52.896
|
|
- type: precision_at_10
|
|
value: 13.8
|
|
- type: precision_at_100
|
|
value: 1.609
|
|
- type: precision_at_1000
|
|
value: 0.166
|
|
- type: precision_at_20
|
|
value: 7.444000000000001
|
|
- type: precision_at_3
|
|
value: 32.623999999999995
|
|
- type: precision_at_5
|
|
value: 23.735
|
|
- type: recall_at_1
|
|
value: 36.525
|
|
- type: recall_at_10
|
|
value: 83.893
|
|
- type: recall_at_100
|
|
value: 96.345
|
|
- type: recall_at_1000
|
|
value: 99.126
|
|
- type: recall_at_20
|
|
value: 89.812
|
|
- type: recall_at_3
|
|
value: 62.58899999999999
|
|
- type: recall_at_5
|
|
value: 73.64500000000001
|
|
task:
|
|
type: Retrieval
|
|
- dataset:
|
|
config: default
|
|
name: MTEB RuReviewsClassification
|
|
revision: f6d2c31f4dc6b88f468552750bfec05b4b41b05a
|
|
split: test
|
|
type: ai-forever/ru-reviews-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 68.2373046875
|
|
- type: f1
|
|
value: 66.6798984937843
|
|
- type: f1_weighted
|
|
value: 66.67858774240374
|
|
- type: main_score
|
|
value: 68.2373046875
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB RuSTSBenchmarkSTS
|
|
revision: 7cf24f325c6da6195df55bef3d86b5e0616f3018
|
|
split: test
|
|
type: ai-forever/ru-stsbenchmark-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 77.06911833905438
|
|
- type: cosine_spearman
|
|
value: 77.84605139753621
|
|
- type: euclidean_pearson
|
|
value: 76.3616511204864
|
|
- type: euclidean_spearman
|
|
value: 77.84487946345095
|
|
- type: main_score
|
|
value: 77.84605139753621
|
|
- type: manhattan_pearson
|
|
value: 76.35303659263998
|
|
- type: manhattan_spearman
|
|
value: 77.87677782965115
|
|
- type: pearson
|
|
value: 77.06911833905438
|
|
- type: spearman
|
|
value: 77.84605139753621
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB RuSciBenchGRNTIClassification
|
|
revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1
|
|
split: test
|
|
type: ai-forever/ru-scibench-grnti-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 61.23535156249999
|
|
- type: f1
|
|
value: 59.029291161802334
|
|
- type: f1_weighted
|
|
value: 59.041548793589406
|
|
- type: main_score
|
|
value: 61.23535156249999
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB RuSciBenchGRNTIClusteringP2P
|
|
revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1
|
|
split: test
|
|
type: ai-forever/ru-scibench-grnti-classification
|
|
metrics:
|
|
- type: main_score
|
|
value: 56.82815630686135
|
|
- type: v_measure
|
|
value: 56.82815630686135
|
|
- type: v_measure_std
|
|
value: 0.6871068462323323
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: default
|
|
name: MTEB RuSciBenchOECDClassification
|
|
revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471
|
|
split: test
|
|
type: ai-forever/ru-scibench-oecd-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 48.1005859375
|
|
- type: f1
|
|
value: 44.918516110124315
|
|
- type: f1_weighted
|
|
value: 44.91942618115105
|
|
- type: main_score
|
|
value: 48.1005859375
|
|
task:
|
|
type: Classification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB RuSciBenchOECDClusteringP2P
|
|
revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471
|
|
split: test
|
|
type: ai-forever/ru-scibench-oecd-classification
|
|
metrics:
|
|
- type: main_score
|
|
value: 48.72707742931753
|
|
- type: v_measure
|
|
value: 48.72707742931753
|
|
- type: v_measure_std
|
|
value: 0.7258468439420995
|
|
task:
|
|
type: Clustering
|
|
- dataset:
|
|
config: ru
|
|
name: MTEB STS22 (ru)
|
|
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
|
split: test
|
|
type: mteb/sts22-crosslingual-sts
|
|
metrics:
|
|
- type: cosine_pearson
|
|
value: 64.95220904597029
|
|
- type: cosine_spearman
|
|
value: 67.35282990065247
|
|
- type: euclidean_pearson
|
|
value: 64.72045496418937
|
|
- type: euclidean_spearman
|
|
value: 67.35282990065247
|
|
- type: main_score
|
|
value: 67.35282990065247
|
|
- type: manhattan_pearson
|
|
value: 64.40621455763392
|
|
- type: manhattan_spearman
|
|
value: 66.99408273892949
|
|
- type: pearson
|
|
value: 64.95220904597029
|
|
- type: spearman
|
|
value: 67.35282990065247
|
|
task:
|
|
type: STS
|
|
- dataset:
|
|
config: default
|
|
name: MTEB SensitiveTopicsClassification
|
|
revision: 416b34a802308eac30e4192afc0ff99bb8dcc7f2
|
|
split: test
|
|
type: ai-forever/sensitive-topics-classification
|
|
metrics:
|
|
- type: accuracy
|
|
value: 29.624023437500004
|
|
- type: f1
|
|
value: 33.214028020582894
|
|
- type: lrap
|
|
value: 44.53599717881868
|
|
- type: main_score
|
|
value: 29.624023437500004
|
|
task:
|
|
type: MultilabelClassification
|
|
- dataset:
|
|
config: default
|
|
name: MTEB TERRa
|
|
revision: 7b58f24536063837d644aab9a023c62199b2a612
|
|
split: dev
|
|
type: ai-forever/terra-pairclassification
|
|
metrics:
|
|
- type: cosine_accuracy
|
|
value: 57.98045602605863
|
|
- type: cosine_accuracy_threshold
|
|
value: 83.04829597473145
|
|
- type: cosine_ap
|
|
value: 55.56580974377611
|
|
- type: cosine_f1
|
|
value: 66.9603524229075
|
|
- type: cosine_f1_threshold
|
|
value: 73.216313123703
|
|
- type: cosine_precision
|
|
value: 50.498338870431894
|
|
- type: cosine_recall
|
|
value: 99.34640522875817
|
|
- type: dot_accuracy
|
|
value: 57.98045602605863
|
|
- type: dot_accuracy_threshold
|
|
value: 83.04829597473145
|
|
- type: dot_ap
|
|
value: 55.56580974377611
|
|
- type: dot_f1
|
|
value: 66.9603524229075
|
|
- type: dot_f1_threshold
|
|
value: 73.21631908416748
|
|
- type: dot_precision
|
|
value: 50.498338870431894
|
|
- type: dot_recall
|
|
value: 99.34640522875817
|
|
- type: euclidean_accuracy
|
|
value: 57.98045602605863
|
|
- type: euclidean_accuracy_threshold
|
|
value: 58.226633071899414
|
|
- type: euclidean_ap
|
|
value: 55.56580974377611
|
|
- type: euclidean_f1
|
|
value: 66.9603524229075
|
|
- type: euclidean_f1_threshold
|
|
value: 73.18969368934631
|
|
- type: euclidean_precision
|
|
value: 50.498338870431894
|
|
- type: euclidean_recall
|
|
value: 99.34640522875817
|
|
- type: main_score
|
|
value: 55.56580974377611
|
|
- type: manhattan_accuracy
|
|
value: 57.98045602605863
|
|
- type: manhattan_accuracy_threshold
|
|
value: 1336.6012573242188
|
|
- type: manhattan_ap
|
|
value: 55.5371135438789
|
|
- type: manhattan_f1
|
|
value: 66.95842450765863
|
|
- type: manhattan_f1_threshold
|
|
value: 1720.5078125
|
|
- type: manhattan_precision
|
|
value: 50.32894736842105
|
|
- type: manhattan_recall
|
|
value: 100.0
|
|
- type: max_ap
|
|
value: 55.56580974377611
|
|
- type: max_f1
|
|
value: 66.9603524229075
|
|
- type: max_precision
|
|
value: 50.498338870431894
|
|
- type: max_recall
|
|
value: 100.0
|
|
- type: similarity_accuracy
|
|
value: 57.98045602605863
|
|
- type: similarity_accuracy_threshold
|
|
value: 83.04829597473145
|
|
- type: similarity_ap
|
|
value: 55.56580974377611
|
|
- type: similarity_f1
|
|
value: 66.9603524229075
|
|
- type: similarity_f1_threshold
|
|
value: 73.216313123703
|
|
- type: similarity_precision
|
|
value: 50.498338870431894
|
|
- type: similarity_recall
|
|
value: 99.34640522875817
|
|
task:
|
|
type: PairClassification
|
|
pipeline_tag: sentence-similarity
|
|
tags:
|
|
- sentence-transformers
|
|
- feature-extraction
|
|
- sentence-similarity
|
|
- mteb
|
|
---
|
|
-
|
|
|
|
<h1 align="center">KaLM-Embedding</h1>
|
|
|
|
|
|
**KaLM-Embedding** is a series of embedding models adapted from auto-regressive LLMs with superior training data.
|
|
|
|
KaLM-embedding-multilingual-mini is trained from [Qwen/Qwen2-0.5B](https://huggingface.co/Qwen/Qwen2-0.5B) with massive weakly-supervised pre-training and supervised fine-tuning data.
|
|
|
|
|
|
## 📑 Open-source Plan
|
|
|
|
- [x] Model Checkpoint
|
|
- [x] [KaLM-embedding-multilingual-mini-v1](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-v1)
|
|
- [x] [KaLM-embedding-multilingual-mini-instruct-v1](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1)
|
|
- [ ] KaLM-embedding-multilingual-max-v1
|
|
- [x] Training and Evaluation Code: [HITsz-TMG/KaLM-Embedding](https://github.com/HITsz-TMG/KaLM-Embedding)
|
|
- [ ] Technical Report (in 2024)
|
|
- [ ] Training Data
|
|
|
|
|
|
## Evaluation
|
|
|
|
| Model Name | Model Size | C-MTEB(35) | MTEB(56) | avg
|
|
|:----:|:---:|:---:|:---:|:---:|
|
|
| [multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) | 560M | 58.81 | 61.5 | 60.16
|
|
| [bge-m3 (dense)](https://huggingface.co/BAAI/bge-m3) | 560M | 60.80 | 59.84 | 60.32
|
|
| [gte-multilingual-base (dense)](https://huggingface.co/Alibaba-NLP/gte-multilingual-base) | **305M** | 62.72 | 61.40 | 62.06
|
|
| [KaLM-embedding-multilingual-mini-v1](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-v1) | 494M | 62.31 | 61.87 | 62.09
|
|
| [KaLM-embedding-multilingual-mini-instruct-v1](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1) | 494M | **63.57** | **64.74** | **64.16**
|
|
|
|
|
|
|
|
## Requirements
|
|
Since we have used the Qwen2 model, we advise you to install `transformers>=4.37.0`, or you might encounter the following error:
|
|
```
|
|
KeyError: 'qwen2'
|
|
```
|
|
|
|
|
|
## Usage
|
|
|
|
Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
|
|
|
|
```
|
|
pip install -U sentence-transformers
|
|
```
|
|
|
|
Then you can use the model like this:
|
|
|
|
```python
|
|
from sentence_transformers import SentenceTransformer
|
|
|
|
|
|
sentences = ["This is an example sentence", "Each sentence is converted"]
|
|
|
|
model = SentenceTransformer('{MODEL_NAME_OR_PATH}') # Do NOT set trust_remote_code
|
|
model.max_seq_length = 512
|
|
|
|
embeddings = model.encode(
|
|
sentences,
|
|
normalize_embeddings=True,
|
|
batch_size=256,
|
|
show_progress_bar=True
|
|
)
|
|
print(embeddings)
|
|
```
|
|
|
|
We add instruction for asymmetric tasks: retrieval, reranking, classification and clustering.
|
|
<!-- We add instruction for classification and clustering. -->
|
|
If you want to add instruction to the query (no instruction for the corpus), you can use the model like this:
|
|
|
|
```python
|
|
from sentence_transformers import SentenceTransformer
|
|
|
|
|
|
sentences = ["This is an example sentence", "Each sentence is converted"]
|
|
|
|
model = SentenceTransformer('{MODEL_NAME_OR_PATH}') # Do NOT set trust_remote_code
|
|
model.max_seq_length = 512
|
|
|
|
prompt = "Instruct: Classifying the category of french news. \n Query: "
|
|
embeddings = model.encode(
|
|
sentences,
|
|
prompt=prompt,
|
|
normalize_embeddings=True,
|
|
batch_size=256,
|
|
show_progress_bar=True
|
|
)
|
|
print(embeddings)
|
|
```
|
|
|
|
|
|
## Contact
|
|
If you encounter any issue, feel free to contact us via the email: yanshek.woo@gmail.com
|