license	model-index	pipeline_tag	tags
mit	name results KaLM-Embedding dataset metrics task config name revision split type en-ext MTEB AmazonCounterfactualClassification (en-ext) e8379541af4e31359cca9fbcf4b00f2671dba205 test mteb/amazon_counterfactual type value accuracy 94.35532233883059 type value ap 60.40219300665376 type value ap_weighted 60.40219300665376 type value f1 86.52001470357649 type value f1_weighted 94.65531755022661 type value main_score 94.35532233883059 type Classification dataset metrics task config name revision split type en MTEB AmazonCounterfactualClassification (en) e8379541af4e31359cca9fbcf4b00f2671dba205 test mteb/amazon_counterfactual type value accuracy 91.71641791044776 type value ap 68.4050364584575 type value ap_weighted 68.4050364584575 type value f1 87.91854774634491 type value f1_weighted 92.0430596057422 type value main_score 91.71641791044776 type Classification dataset metrics task config name revision split type default MTEB AmazonPolarityClassification e2d317d38cd51312af73b3d32a06d1a08b442046 test mteb/amazon_polarity type value accuracy 96.49945000000001 type value ap 94.97348227456295 type value ap_weighted 94.97348227456295 type value f1 96.49855824500423 type value f1_weighted 96.49855824500422 type value main_score 96.49945000000001 type Classification dataset metrics task config name revision split type en MTEB AmazonReviewsClassification (en) 1399c76144fd37290681b995c656ef9b2e06e26d test mteb/amazon_reviews_multi type value accuracy 61.242 type value f1 59.353696237560094 type value f1_weighted 59.35369623756011 type value main_score 61.242 type Classification dataset metrics task config name revision split type default MTEB ArguAna c22ab2a51041ffd869aaddef7af8d8215647e41a test mteb/arguana type value main_score 56.569 type value map_at_1 31.080999999999996 type value map_at_10 47.432 type value map_at_100 48.247 type value map_at_1000 48.251 type value map_at_20 48.114000000000004 type value map_at_3 42.425000000000004 type value map_at_5 45.128 type value mrr_at_1 31.57894736842105 type value mrr_at_10 47.6253132832081 type value mrr_at_100 48.440395388879296 type value mrr_at_1000 48.44416076630039 type value mrr_at_20 48.30706364782469 type value mrr_at_3 42.59127548601235 type value mrr_at_5 45.347321005215804 type value nauc_map_at_1000_diff1 7.110790588301176 type value nauc_map_at_1000_max -12.892696039828866 type value nauc_map_at_1000_std -15.5709273320573 type value nauc_map_at_100_diff1 7.117551663882657 type value nauc_map_at_100_max -12.882680977142957 type value nauc_map_at_100_std -15.56350483617667 type value nauc_map_at_10_diff1 6.903272993199564 type value nauc_map_at_10_max -13.012877497725961 type value nauc_map_at_10_std -15.947400478856006 type value nauc_map_at_1_diff1 10.03503740028087 type value nauc_map_at_1_max -13.351553937797 type value nauc_map_at_1_std -14.137614923859612 type value nauc_map_at_20_diff1 7.01754882034529 type value nauc_map_at_20_max -12.864438636302197 type value nauc_map_at_20_std -15.541510619190976 type value nauc_map_at_3_diff1 7.018587254951812 type value nauc_map_at_3_max -13.38420244471981 type value nauc_map_at_3_std -16.127099270987785 type value nauc_map_at_5_diff1 6.920961668066123 type value nauc_map_at_5_max -13.169892625713931 type value nauc_map_at_5_std -16.21272880801226 type value nauc_mrr_at_1000_diff1 5.5525831294754004 type value nauc_mrr_at_1000_max -12.98089269414052 type value nauc_mrr_at_1000_std -15.396489593627944 type value nauc_mrr_at_100_diff1 5.559525360367539 type value nauc_mrr_at_100_max -12.970885236428334 type value nauc_mrr_at_100_std -15.389102542398783 type value nauc_mrr_at_10_diff1 5.38828048977972 type value nauc_mrr_at_10_max -13.096637253890634 type value nauc_mrr_at_10_std -15.775810422484374 type value nauc_mrr_at_1_diff1 8.58091801149426 type value nauc_mrr_at_1_max -12.352949021555306 type value nauc_mrr_at_1_std -13.545487974417847 type value nauc_mrr_at_20_diff1 5.4666282281067735 type value nauc_mrr_at_20_max -12.952039027828944 type value nauc_mrr_at_20_std -15.367907454271231 type value nauc_mrr_at_3_diff1 5.1862331302405735 type value nauc_mrr_at_3_max -13.816401285559108 type value nauc_mrr_at_3_std -15.872101319770382 type value nauc_mrr_at_5_diff1 5.471097057115419 type value nauc_mrr_at_5_max -13.269134531334442 type value nauc_mrr_at_5_std -15.95735511276538 type value nauc_ndcg_at_1000_diff1 6.8032235432235275 type value nauc_ndcg_at_1000_max -12.52617810408163 type value nauc_ndcg_at_1000_std -15.38677998208727 type value nauc_ndcg_at_100_diff1 6.971743190062509 type value nauc_ndcg_at_100_max -12.284060222136334 type value nauc_ndcg_at_100_std -15.203583619739097 type value nauc_ndcg_at_10_diff1 5.9423315360857005 type value nauc_ndcg_at_10_max -12.649746010742199 type value nauc_ndcg_at_10_std -16.72153869758235 type value nauc_ndcg_at_1_diff1 10.03503740028087 type value nauc_ndcg_at_1_max -13.351553937797 type value nauc_ndcg_at_1_std -14.137614923859612 type value nauc_ndcg_at_20_diff1 6.379802915097805 type value nauc_ndcg_at_20_max -12.01427315352701 type value nauc_ndcg_at_20_std -15.108250307425825 type value nauc_ndcg_at_3_diff1 6.298556094258956 type value nauc_ndcg_at_3_max -13.536187803253377 type value nauc_ndcg_at_3_std -16.999347732797407 type value nauc_ndcg_at_5_diff1 6.099858591554027 type value nauc_ndcg_at_5_max -13.097631098081774 type value nauc_ndcg_at_5_std -17.215525664264348 type value nauc_precision_at_1000_diff1 -21.130247827110427 type value nauc_precision_at_1000_max 24.21748822806628 type value nauc_precision_at_1000_std 83.6578697460551 type value nauc_precision_at_100_diff1 29.395727608507894 type value nauc_precision_at_100_max 51.676651935775695 type value nauc_precision_at_100_std 62.92260397258278 type value nauc_precision_at_10_diff1 -0.25306953208178373 type value nauc_precision_at_10_max -9.710491261292093 type value nauc_precision_at_10_std -21.697648668302183 type value nauc_precision_at_1_diff1 10.03503740028087 type value nauc_precision_at_1_max -13.351553937797 type value nauc_precision_at_1_std -14.137614923859612 type value nauc_precision_at_20_diff1 -2.084669856957687 type value nauc_precision_at_20_max 6.736841084303921 type value nauc_precision_at_20_std -0.330152716888139 type value nauc_precision_at_3_diff1 4.202256387521114 type value nauc_precision_at_3_max -14.043068948669681 type value nauc_precision_at_3_std -19.71625712734227 type value nauc_precision_at_5_diff1 3.2694130100522667 type value nauc_precision_at_5_max -12.7772753118202 type value nauc_precision_at_5_std -20.917228577779888 type value nauc_recall_at_1000_diff1 -21.13024782711332 type value nauc_recall_at_1000_max 24.21748822806101 type value nauc_recall_at_1000_std 83.6578697460535 type value nauc_recall_at_100_diff1 29.395727608504448 type value nauc_recall_at_100_max 51.67665193577227 type value nauc_recall_at_100_std 62.92260397258032 type value nauc_recall_at_10_diff1 -0.2530695320818313 type value nauc_recall_at_10_max -9.710491261292015 type value nauc_recall_at_10_std -21.697648668302048 type value nauc_recall_at_1_diff1 10.03503740028087 type value nauc_recall_at_1_max -13.351553937797 type value nauc_recall_at_1_std -14.137614923859612 type value nauc_recall_at_20_diff1 -2.0846698569576856 type value nauc_recall_at_20_max 6.736841084303534 type value nauc_recall_at_20_std -0.3301527168878837 type value nauc_recall_at_3_diff1 4.202256387521115 type value nauc_recall_at_3_max -14.043068948669694 type value nauc_recall_at_3_std -19.716257127342317 type value nauc_recall_at_5_diff1 3.26941301005235 type value nauc_recall_at_5_max -12.777275311820102 type value nauc_recall_at_5_std -20.917228577779866 type value ndcg_at_1 31.080999999999996 type value ndcg_at_10 56.569 type value ndcg_at_100 59.772999999999996 type value ndcg_at_1000 59.843 type value ndcg_at_20 58.933 type value ndcg_at_3 46.209 type value ndcg_at_5 51.090999999999994 type value precision_at_1 31.080999999999996 type value precision_at_10 8.578 type value precision_at_100 0.991 type value precision_at_1000 0.1 type value precision_at_20 4.744000000000001 type value precision_at_3 19.061 type value precision_at_5 13.812 type value recall_at_1 31.080999999999996 type value recall_at_10 85.775 type value recall_at_100 99.14699999999999 type value recall_at_1000 99.644 type value recall_at_20 94.879 type value recall_at_3 57.18299999999999 type value recall_at_5 69.06099999999999 type Retrieval dataset metrics task config name revision split type default MTEB ArxivClusteringP2P a122ad7f3f0291bf49cc6f4d32aa80929df69d5d test mteb/arxiv-clustering-p2p type value main_score 48.009758343820856 type value v_measure 48.009758343820856 type value v_measure_std 14.203651443985635 type Clustering dataset metrics task config name revision split type default MTEB ArxivClusteringS2S f910caf1a6075f7329cdf8c1a6135696f37dbd53 test mteb/arxiv-clustering-s2s type value main_score 39.401811401341035 type value v_measure 39.401811401341035 type value v_measure_std 14.736655369522248 type Clustering dataset metrics task config name revision split type default MTEB AskUbuntuDupQuestions 2000358ca161889fa9c082cb41daa8dcfb161a54 test mteb/askubuntudupquestions-reranking type value main_score 60.158996366210474 type value map 60.158996366210474 type value mrr 74.69034428175702 type value nAUC_map_diff1 7.7660414737755605 type value nAUC_map_max 20.377348037855818 type value nAUC_map_std 18.290516035806565 type value nAUC_mrr_diff1 10.721266751736124 type value nAUC_mrr_max 31.3686330442438 type value nAUC_mrr_std 19.240868443170196 type Reranking dataset metrics task config name revision split type default MTEB BIOSSES d3fb88f8f02e40887cd149695127462bbcf29b4a test mteb/biosses-sts type value cosine_pearson 87.53887478826596 type value cosine_spearman 86.32606338345799 type value euclidean_pearson 86.76233071291158 type value euclidean_spearman 86.32606338345799 type value main_score 86.32606338345799 type value manhattan_pearson 86.05455915524152 type value manhattan_spearman 85.8868967502423 type value pearson 87.53887478826596 type value spearman 86.32606338345799 type STS dataset metrics task config name revision split type default MTEB Banking77Classification 0fd18e25b25c072e09e0d92ab615fda904d66300 test mteb/banking77 type value accuracy 84.92857142857144 type value f1 84.30505630131526 type value f1_weighted 84.30505630131528 type value main_score 84.92857142857144 type Classification dataset metrics task config name revision split type default MTEB BiorxivClusteringP2P 65b79d1d13f80053f67aca9498d9402c2d9f1f40 test mteb/biorxiv-clustering-p2p type value main_score 40.014867273983484 type value v_measure 40.014867273983484 type value v_measure_std 0.6558905123714063 type Clustering dataset metrics task config name revision split type default MTEB BiorxivClusteringS2S 258694dd0231531bc1fd9de6ceb52a0853c6d908 test mteb/biorxiv-clustering-s2s type value main_score 33.79424302438114 type value v_measure 33.79424302438114 type value v_measure_std 0.837779778459544 type Clustering dataset metrics task config name revision split type default MTEB CQADupstackAndroidRetrieval f46a197baaae43b4f621051089b82a364682dfeb test mteb/cqadupstack-android type value main_score 52.884 type value map_at_1 34.634 type value map_at_10 46.339000000000006 type value map_at_100 47.857 type value map_at_1000 47.97 type value map_at_20 47.205000000000005 type value map_at_3 42.543 type value map_at_5 44.772 type value mrr_at_1 41.3447782546495 type value mrr_at_10 52.23857210981671 type value mrr_at_100 52.90915062899396 type value mrr_at_1000 52.95240146583995 type value mrr_at_20 52.655345331835804 type value mrr_at_3 49.71387696709583 type value mrr_at_5 51.23748211731041 type value nauc_map_at_1000_diff1 51.49705927936061 type value nauc_map_at_1000_max 35.528845247090466 type value nauc_map_at_1000_std -4.253741985593714 type value nauc_map_at_100_diff1 51.508584685268886 type value nauc_map_at_100_max 35.56248075672379 type value nauc_map_at_100_std -4.176500881199186 type value nauc_map_at_10_diff1 51.338718973920614 type value nauc_map_at_10_max 34.946543347441214 type value nauc_map_at_10_std -5.33037717427031 type value nauc_map_at_1_diff1 56.23620820617472 type value nauc_map_at_1_max 30.320970401424987 type value nauc_map_at_1_std -6.655365474007067 type value nauc_map_at_20_diff1 51.51775947048102 type value nauc_map_at_20_max 35.25983470448141 type value nauc_map_at_20_std -4.612859125963163 type value nauc_map_at_3_diff1 52.725269902770755 type value nauc_map_at_3_max 33.299803481018195 type value nauc_map_at_3_std -6.33353874546021 type value nauc_map_at_5_diff1 51.672349084315485 type value nauc_map_at_5_max 34.645370794379886 type value nauc_map_at_5_std -5.94117791112353 type value nauc_mrr_at_1000_diff1 49.12249635354981 type value nauc_mrr_at_1000_max 36.29480359532615 type value nauc_mrr_at_1000_std -4.665759763477847 type value nauc_mrr_at_100_diff1 49.11255442003998 type value nauc_mrr_at_100_max 36.29806935257465 type value nauc_mrr_at_100_std -4.663481407381479 type value nauc_mrr_at_10_diff1 48.779673215220065 type value nauc_mrr_at_10_max 36.12766214960087 type value nauc_mrr_at_10_std -5.0778877090392625 type value nauc_mrr_at_1_diff1 53.70331290003521 type value nauc_mrr_at_1_max 35.17671705244682 type value nauc_mrr_at_1_std -6.289432335416569 type value nauc_mrr_at_20_diff1 48.98440189775321 type value nauc_mrr_at_20_max 36.24567442841102 type value nauc_mrr_at_20_std -4.808524080549843 type value nauc_mrr_at_3_diff1 50.09142180621504 type value nauc_mrr_at_3_max 36.57201237478509 type value nauc_mrr_at_3_std -5.10258589719658 type value nauc_mrr_at_5_diff1 49.15413181233011 type value nauc_mrr_at_5_max 36.64387975128488 type value nauc_mrr_at_5_std -5.2142664019104 type value nauc_ndcg_at_1000_diff1 49.48338541267995 type value nauc_ndcg_at_1000_max 36.71225124867686 type value nauc_ndcg_at_1000_std -2.1565353674328636 type value nauc_ndcg_at_100_diff1 49.378803009143354 type value nauc_ndcg_at_100_max 37.05072158645242 type value nauc_ndcg_at_100_std -1.1554881315239078 type value nauc_ndcg_at_10_diff1 48.217255194293706 type value nauc_ndcg_at_10_max 35.70709987917217 type value nauc_ndcg_at_10_std -4.5843409864100835 type value nauc_ndcg_at_1_diff1 53.70331290003521 type value nauc_ndcg_at_1_max 35.17671705244682 type value nauc_ndcg_at_1_std -6.289432335416569 type value nauc_ndcg_at_20_diff1 48.90479671421663 type value nauc_ndcg_at_20_max 35.63061062961699 type value nauc_ndcg_at_20_std -3.2759049624453924 type value nauc_ndcg_at_3_diff1 50.66992100707998 type value nauc_ndcg_at_3_max 35.647144096807054 type value nauc_ndcg_at_3_std -4.675684277632912 type value nauc_ndcg_at_5_diff1 48.86023024957704 type value nauc_ndcg_at_5_max 36.36204191994049 type value nauc_ndcg_at_5_std -4.979721506683613 type value nauc_precision_at_1000_diff1 -20.176146428291695 type value nauc_precision_at_1000_max -4.944333530911747 type value nauc_precision_at_1000_std -2.6416464331580256 type value nauc_precision_at_100_diff1 -11.455305661135391 type value nauc_precision_at_100_max 9.563783942313348 type value nauc_precision_at_100_std 9.987888995757324 type value nauc_precision_at_10_diff1 6.577302086017673 type value nauc_precision_at_10_max 25.67586949524924 type value nauc_precision_at_10_std 5.543682394632135 type value nauc_precision_at_1_diff1 53.70331290003521 type value nauc_precision_at_1_max 35.17671705244682 type value nauc_precision_at_1_std -6.289432335416569 type value nauc_precision_at_20_diff1 0.0352451246393809 type value nauc_precision_at_20_max 19.02340589034973 type value nauc_precision_at_20_std 10.156322995661567 type value nauc_precision_at_3_diff1 31.114868446262108 type value nauc_precision_at_3_max 35.740653736733925 type value nauc_precision_at_3_std -0.4754489918596968 type value nauc_precision_at_5_diff1 17.05966182310583 type value nauc_precision_at_5_max 32.37346687203089 type value nauc_precision_at_5_std 1.4954175443689899 type value nauc_recall_at_1000_diff1 42.86116448480766 type value nauc_recall_at_1000_max 63.759509563968976 type value nauc_recall_at_1000_std 61.175429354991614 type value nauc_recall_at_100_diff1 40.88375670987642 type value nauc_recall_at_100_max 44.62608189829668 type value nauc_recall_at_100_std 25.55163256804942 type value nauc_recall_at_10_diff1 37.759771219935175 type value nauc_recall_at_10_max 31.146081092167627 type value nauc_recall_at_10_std -4.512890345394815 type value nauc_recall_at_1_diff1 56.23620820617472 type value nauc_recall_at_1_max 30.320970401424987 type value nauc_recall_at_1_std -6.655365474007067 type value nauc_recall_at_20_diff1 38.4827047216752 type value nauc_recall_at_20_max 30.50125803520275 type value nauc_recall_at_20_std 0.8771358044937425 type value nauc_recall_at_3_diff1 47.487079446530906 type value nauc_recall_at_3_max 32.19896007873808 type value nauc_recall_at_3_std -5.164803420738882 type value nauc_recall_at_5_diff1 41.699415045286415 type value nauc_recall_at_5_max 33.168829040464196 type value nauc_recall_at_5_std -5.366546702094067 type value ndcg_at_1 41.345 type value ndcg_at_10 52.884 type value ndcg_at_100 57.94200000000001 type value ndcg_at_1000 59.68 type value ndcg_at_20 54.957 type value ndcg_at_3 47.692 type value ndcg_at_5 50.251000000000005 type value precision_at_1 41.345 type value precision_at_10 10.057 type value precision_at_100 1.574 type value precision_at_1000 0.201 type value precision_at_20 5.9799999999999995 type value precision_at_3 22.842000000000002 type value precision_at_5 16.595 type value recall_at_1 34.634 type value recall_at_10 65.185 type value recall_at_100 85.703 type value recall_at_1000 96.65599999999999 type value recall_at_20 72.322 type value recall_at_3 50.182 type value recall_at_5 57.159000000000006 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackEnglishRetrieval ad9991cb51e31e31e430383c75ffb2885547b5f0 test mteb/cqadupstack-english type value main_score 48.264 type value map_at_1 31.224 type value map_at_10 42.332 type value map_at_100 43.533 type value map_at_1000 43.662 type value map_at_20 42.972 type value map_at_3 39.159 type value map_at_5 41.047 type value mrr_at_1 39.04458598726115 type value mrr_at_10 48.18686179354971 type value mrr_at_100 48.803902946647234 type value mrr_at_1000 48.84702137486075 type value mrr_at_20 48.56368295512913 type value mrr_at_3 45.83864118895968 type value mrr_at_5 47.20806794055207 type value nauc_map_at_1000_diff1 51.86414274615986 type value nauc_map_at_1000_max 34.717053941484025 type value nauc_map_at_1000_std -4.340680651811943 type value nauc_map_at_100_diff1 51.84970191815774 type value nauc_map_at_100_max 34.64676814212115 type value nauc_map_at_100_std -4.4387297635880385 type value nauc_map_at_10_diff1 52.119436277416945 type value nauc_map_at_10_max 33.94135075756255 type value nauc_map_at_10_std -5.625757602694689 type value nauc_map_at_1_diff1 57.92845763299044 type value nauc_map_at_1_max 29.3199164115535 type value nauc_map_at_1_std -11.586283921183611 type value nauc_map_at_20_diff1 51.92734614822424 type value nauc_map_at_20_max 34.35250084161699 type value nauc_map_at_20_std -5.049283716884917 type value nauc_map_at_3_diff1 52.783776450874356 type value nauc_map_at_3_max 32.394255917655535 type value nauc_map_at_3_std -8.05902730660978 type value nauc_map_at_5_diff1 52.14993873615333 type value nauc_map_at_5_max 33.48431923578608 type value nauc_map_at_5_std -6.472903440360678 type value nauc_mrr_at_1000_diff1 50.49829531271091 type value nauc_mrr_at_1000_max 37.183131918098425 type value nauc_mrr_at_1000_std 0.4928095353543418 type value nauc_mrr_at_100_diff1 50.494636141021424 type value nauc_mrr_at_100_max 37.185446950719715 type value nauc_mrr_at_100_std 0.5056844413835279 type value nauc_mrr_at_10_diff1 50.55418166759066 type value nauc_mrr_at_10_max 37.17369235180479 type value nauc_mrr_at_10_std 0.3511264489316608 type value nauc_mrr_at_1_diff1 55.09381247060509 type value nauc_mrr_at_1_max 37.17089033507927 type value nauc_mrr_at_1_std -2.545073558300969 type value nauc_mrr_at_20_diff1 50.46232349188045 type value nauc_mrr_at_20_max 37.22028938157565 type value nauc_mrr_at_20_std 0.4342508184428254 type value nauc_mrr_at_3_diff1 50.98797216868357 type value nauc_mrr_at_3_max 37.32821622965925 type value nauc_mrr_at_3_std -0.6918122573096884 type value nauc_mrr_at_5_diff1 50.477903924122025 type value nauc_mrr_at_5_max 37.343161615517296 type value nauc_mrr_at_5_std 0.34187371397979793 type value nauc_ndcg_at_1000_diff1 49.71083273417971 type value nauc_ndcg_at_1000_max 36.08714449707927 type value nauc_ndcg_at_1000_std 0.3359295264579242 type value nauc_ndcg_at_100_diff1 49.64047591726873 type value nauc_ndcg_at_100_max 36.0502827680962 type value nauc_ndcg_at_100_std 0.4394585830222923 type value nauc_ndcg_at_10_diff1 50.3895028633975 type value nauc_ndcg_at_10_max 35.51838515595454 type value nauc_ndcg_at_10_std -1.8340842845181509 type value nauc_ndcg_at_1_diff1 55.09381247060509 type value nauc_ndcg_at_1_max 37.17089033507927 type value nauc_ndcg_at_1_std -2.545073558300969 type value nauc_ndcg_at_20_diff1 49.975850062007375 type value nauc_ndcg_at_20_max 35.8777155711073 type value nauc_ndcg_at_20_std -1.1833564484981665 type value nauc_ndcg_at_3_diff1 50.3823214340417 type value nauc_ndcg_at_3_max 35.776477162991746 type value nauc_ndcg_at_3_std -3.0969092422279623 type value nauc_ndcg_at_5_diff1 50.18424405483706 type value nauc_ndcg_at_5_max 35.886540678742485 type value nauc_ndcg_at_5_std -2.2048728336054912 type value nauc_precision_at_1000_diff1 -8.409825453277659 type value nauc_precision_at_1000_max 14.148796859940632 type value nauc_precision_at_1000_std 28.34712816378856 type value nauc_precision_at_100_diff1 -4.133099395945424 type value nauc_precision_at_100_max 23.436894225838895 type value nauc_precision_at_100_std 31.777687917658554 type value nauc_precision_at_10_diff1 12.456499608847746 type value nauc_precision_at_10_max 34.40385767678226 type value nauc_precision_at_10_std 22.64168731207244 type value nauc_precision_at_1_diff1 55.09381247060509 type value nauc_precision_at_1_max 37.17089033507927 type value nauc_precision_at_1_std -2.545073558300969 type value nauc_precision_at_20_diff1 4.838516065171166 type value nauc_precision_at_20_max 31.381417947568412 type value nauc_precision_at_20_std 26.974660907322917 type value nauc_precision_at_3_diff1 28.180760599976384 type value nauc_precision_at_3_max 36.40321247194992 type value nauc_precision_at_3_std 9.375871028699667 type value nauc_precision_at_5_diff1 19.689988735115058 type value nauc_precision_at_5_max 35.98837508752083 type value nauc_precision_at_5_std 16.284464606894232 type value nauc_recall_at_1000_diff1 33.594125915695884 type value nauc_recall_at_1000_max 31.574941156196807 type value nauc_recall_at_1000_std 20.460707032380316 type value nauc_recall_at_100_diff1 38.54327301097089 type value nauc_recall_at_100_max 33.368528599783126 type value nauc_recall_at_100_std 15.321500393966641 type value nauc_recall_at_10_diff1 44.219731053687255 type value nauc_recall_at_10_max 31.484342080988824 type value nauc_recall_at_10_std 0.22452148883121484 type value nauc_recall_at_1_diff1 57.92845763299044 type value nauc_recall_at_1_max 29.3199164115535 type value nauc_recall_at_1_std -11.586283921183611 type value nauc_recall_at_20_diff1 41.39285600168573 type value nauc_recall_at_20_max 32.966202138611465 type value nauc_recall_at_20_std 3.365583403518244 type value nauc_recall_at_3_diff1 47.33546382576856 type value nauc_recall_at_3_max 30.988541475501425 type value nauc_recall_at_3_std -5.87940259105687 type value nauc_recall_at_5_diff1 45.27313627261692 type value nauc_recall_at_5_max 32.34545008582682 type value nauc_recall_at_5_std -1.6738776274622713 type value ndcg_at_1 39.045 type value ndcg_at_10 48.264 type value ndcg_at_100 52.493 type value ndcg_at_1000 54.457 type value ndcg_at_20 49.888 type value ndcg_at_3 43.86 type value ndcg_at_5 45.983000000000004 type value precision_at_1 39.045 type value precision_at_10 9.096 type value precision_at_100 1.442 type value precision_at_1000 0.191 type value precision_at_20 5.309 type value precision_at_3 21.316 type value precision_at_5 15.197 type value recall_at_1 31.224 type value recall_at_10 59.080999999999996 type value recall_at_100 76.897 type value recall_at_1000 89.23 type value recall_at_20 64.891 type value recall_at_3 46.076 type value recall_at_5 51.964 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackGamingRetrieval 4885aa143210c98657558c04aaf3dc47cfb54340 test mteb/cqadupstack-gaming type value main_score 62.366 type value map_at_1 42.703 type value map_at_10 56.281000000000006 type value map_at_100 57.260999999999996 type value map_at_1000 57.30800000000001 type value map_at_20 56.871 type value map_at_3 52.897000000000006 type value map_at_5 54.773 type value mrr_at_1 48.589341692789965 type value mrr_at_10 59.43538836642291 type value mrr_at_100 59.999373625798235 type value mrr_at_1000 60.02341349127948 type value mrr_at_20 59.78236245014694 type value mrr_at_3 56.99059561128534 type value mrr_at_5 58.373040752351216 type value nauc_map_at_1000_diff1 51.724911969542475 type value nauc_map_at_1000_max 31.59720256654406 type value nauc_map_at_1000_std -8.448863423330733 type value nauc_map_at_100_diff1 51.721207885585294 type value nauc_map_at_100_max 31.598189555174677 type value nauc_map_at_100_std -8.415293705149518 type value nauc_map_at_10_diff1 51.74316546903847 type value nauc_map_at_10_max 31.370796021816087 type value nauc_map_at_10_std -9.144187110965651 type value nauc_map_at_1_diff1 55.602123379999405 type value nauc_map_at_1_max 26.15423784568626 type value nauc_map_at_1_std -11.354042579102689 type value nauc_map_at_20_diff1 51.71343659271482 type value nauc_map_at_20_max 31.53988815092091 type value nauc_map_at_20_std -8.65212495986148 type value nauc_map_at_3_diff1 52.064639443577846 type value nauc_map_at_3_max 30.3485604522721 type value nauc_map_at_3_std -10.751274075635509 type value nauc_map_at_5_diff1 51.72321940513861 type value nauc_map_at_5_max 30.392319659455435 type value nauc_map_at_5_std -9.939778501885101 type value nauc_mrr_at_1000_diff1 51.184984251728025 type value nauc_mrr_at_1000_max 32.69216958808548 type value nauc_mrr_at_1000_std -8.500776574802599 type value nauc_mrr_at_100_diff1 51.17941032241811 type value nauc_mrr_at_100_max 32.70608756736136 type value nauc_mrr_at_100_std -8.477679942920167 type value nauc_mrr_at_10_diff1 51.07904444322852 type value nauc_mrr_at_10_max 32.65962497893277 type value nauc_mrr_at_10_std -8.709383804816481 type value nauc_mrr_at_1_diff1 54.53142920528978 type value nauc_mrr_at_1_max 30.926785799334677 type value nauc_mrr_at_1_std -10.41145527848442 type value nauc_mrr_at_20_diff1 51.14693383001116 type value nauc_mrr_at_20_max 32.73093259139165 type value nauc_mrr_at_20_std -8.447633887171534 type value nauc_mrr_at_3_diff1 51.17432400675771 type value nauc_mrr_at_3_max 32.85252288214242 type value nauc_mrr_at_3_std -9.21642979066159 type value nauc_mrr_at_5_diff1 51.036935248981905 type value nauc_mrr_at_5_max 32.502626235077095 type value nauc_mrr_at_5_std -8.948887571702919 type value nauc_ndcg_at_1000_diff1 50.73024891705996 type value nauc_ndcg_at_1000_max 33.26584662078177 type value nauc_ndcg_at_1000_std -6.163854205845618 type value nauc_ndcg_at_100_diff1 50.67040290788501 type value nauc_ndcg_at_100_max 33.68165097437155 type value nauc_ndcg_at_100_std -5.301942481514177 type value nauc_ndcg_at_10_diff1 50.407269736351054 type value nauc_ndcg_at_10_max 33.1723247102446 type value nauc_ndcg_at_10_std -7.313191608002288 type value nauc_ndcg_at_1_diff1 54.53142920528978 type value nauc_ndcg_at_1_max 30.926785799334677 type value nauc_ndcg_at_1_std -10.41145527848442 type value nauc_ndcg_at_20_diff1 50.45722009686969 type value nauc_ndcg_at_20_max 33.54250850995858 type value nauc_ndcg_at_20_std -6.008420175252642 type value nauc_ndcg_at_3_diff1 50.769657622259686 type value nauc_ndcg_at_3_max 31.792120043553002 type value nauc_ndcg_at_3_std -10.040327445335686 type value nauc_ndcg_at_5_diff1 50.398976656987614 type value nauc_ndcg_at_5_max 31.61780666125045 type value nauc_ndcg_at_5_std -8.943124136769121 type value nauc_precision_at_1000_diff1 -17.275717791952 type value nauc_precision_at_1000_max 7.275527027803384 type value nauc_precision_at_1000_std 16.685486896410826 type value nauc_precision_at_100_diff1 -11.162266422032406 type value nauc_precision_at_100_max 12.70258577369679 type value nauc_precision_at_100_std 21.391285680664513 type value nauc_precision_at_10_diff1 7.81828602576801 type value nauc_precision_at_10_max 24.78598247621288 type value nauc_precision_at_10_std 9.374021745818432 type value nauc_precision_at_1_diff1 54.53142920528978 type value nauc_precision_at_1_max 30.926785799334677 type value nauc_precision_at_1_std -10.41145527848442 type value nauc_precision_at_20_diff1 0.1631191398252266 type value nauc_precision_at_20_max 20.619391150501272 type value nauc_precision_at_20_std 16.276264697116872 type value nauc_precision_at_3_diff1 27.04714503298839 type value nauc_precision_at_3_max 30.101606964258337 type value nauc_precision_at_3_std -3.681729229946907 type value nauc_precision_at_5_diff1 17.843974173274304 type value nauc_precision_at_5_max 25.676881643654763 type value nauc_precision_at_5_std 1.5965157990195873 type value nauc_recall_at_1000_diff1 29.087262485289735 type value nauc_recall_at_1000_max 59.55059060998873 type value nauc_recall_at_1000_std 62.21218125216127 type value nauc_recall_at_100_diff1 41.30594954847261 type value nauc_recall_at_100_max 48.03865105456248 type value nauc_recall_at_100_std 28.904820877938946 type value nauc_recall_at_10_diff1 43.528832373563795 type value nauc_recall_at_10_max 36.333747103215266 type value nauc_recall_at_10_std -0.586937217589867 type value nauc_recall_at_1_diff1 55.602123379999405 type value nauc_recall_at_1_max 26.15423784568626 type value nauc_recall_at_1_std -11.354042579102689 type value nauc_recall_at_20_diff1 42.86486871096986 type value nauc_recall_at_20_max 39.37052680687811 type value nauc_recall_at_20_std 7.7270172598031985 type value nauc_recall_at_3_diff1 46.744057097749746 type value nauc_recall_at_3_max 32.0901543978326 type value nauc_recall_at_3_std -9.836059759091158 type value nauc_recall_at_5_diff1 44.52443640046374 type value nauc_recall_at_5_max 31.155871822952808 type value nauc_recall_at_5_std -7.116612032547676 type value ndcg_at_1 48.589 type value ndcg_at_10 62.366 type value ndcg_at_100 66.011 type value ndcg_at_1000 66.88199999999999 type value ndcg_at_20 63.979 type value ndcg_at_3 56.764 type value ndcg_at_5 59.426 type value precision_at_1 48.589 type value precision_at_10 9.981 type value precision_at_100 1.277 type value precision_at_1000 0.13899999999999998 type value precision_at_20 5.514 type value precision_at_3 25.308000000000003 type value precision_at_5 17.241 type value recall_at_1 42.703 type value recall_at_10 77.08 type value recall_at_100 92.374 type value recall_at_1000 98.402 type value recall_at_20 82.87400000000001 type value recall_at_3 62.138000000000005 type value recall_at_5 68.679 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackGisRetrieval 5003b3064772da1887988e05400cf3806fe491f2 test mteb/cqadupstack-gis type value main_score 39.971000000000004 type value map_at_1 25.06 type value map_at_10 34.551 type value map_at_100 35.568 type value map_at_1000 35.65 type value map_at_20 35.127 type value map_at_3 31.936999999999998 type value map_at_5 33.186 type value mrr_at_1 27.11864406779661 type value mrr_at_10 36.72652676889963 type value mrr_at_100 37.57204686098606 type value mrr_at_1000 37.63141267969674 type value mrr_at_20 37.19310670147632 type value mrr_at_3 34.27495291902072 type value mrr_at_5 35.438794726930304 type value nauc_map_at_1000_diff1 43.63829107634628 type value nauc_map_at_1000_max 23.954060999822257 type value nauc_map_at_1000_std -0.5807446969781898 type value nauc_map_at_100_diff1 43.610748406014466 type value nauc_map_at_100_max 23.94949736158448 type value nauc_map_at_100_std -0.5982601848367343 type value nauc_map_at_10_diff1 43.72900243122612 type value nauc_map_at_10_max 23.508469522079885 type value nauc_map_at_10_std -0.5258931194184133 type value nauc_map_at_1_diff1 50.922871467903654 type value nauc_map_at_1_max 24.6067671408884 type value nauc_map_at_1_std -4.630126214452492 type value nauc_map_at_20_diff1 43.63024854824786 type value nauc_map_at_20_max 23.874524344212734 type value nauc_map_at_20_std -0.556366665388133 type value nauc_map_at_3_diff1 44.38253552931588 type value nauc_map_at_3_max 22.561513802056236 type value nauc_map_at_3_std -3.005119773408719 type value nauc_map_at_5_diff1 44.016586535650795 type value nauc_map_at_5_max 23.302456735449038 type value nauc_map_at_5_std -1.7618309245289323 type value nauc_mrr_at_1000_diff1 42.68205493907015 type value nauc_mrr_at_1000_max 26.024690905326025 type value nauc_mrr_at_1000_std 0.6287706252427459 type value nauc_mrr_at_100_diff1 42.654961103491004 type value nauc_mrr_at_100_max 26.029087860328065 type value nauc_mrr_at_100_std 0.6163052064323858 type value nauc_mrr_at_10_diff1 42.56564515109072 type value nauc_mrr_at_10_max 25.666414824261224 type value nauc_mrr_at_10_std 0.7949641234835698 type value nauc_mrr_at_1_diff1 49.966125488185206 type value nauc_mrr_at_1_max 27.193710462071348 type value nauc_mrr_at_1_std -2.2786990240033718 type value nauc_mrr_at_20_diff1 42.65274684886744 type value nauc_mrr_at_20_max 26.052180768841172 type value nauc_mrr_at_20_std 0.7171447318848092 type value nauc_mrr_at_3_diff1 43.22408289408012 type value nauc_mrr_at_3_max 25.34061478734211 type value nauc_mrr_at_3_std -1.1093305128661515 type value nauc_mrr_at_5_diff1 42.87983482470224 type value nauc_mrr_at_5_max 25.91557396366082 type value nauc_mrr_at_5_std -0.13066697110897257 type value nauc_ndcg_at_1000_diff1 41.53426396594562 type value nauc_ndcg_at_1000_max 25.526814765685046 type value nauc_ndcg_at_1000_std 2.2841859589382487 type value nauc_ndcg_at_100_diff1 40.61825803826763 type value nauc_ndcg_at_100_max 25.344384823963455 type value nauc_ndcg_at_100_std 1.9818508179504288 type value nauc_ndcg_at_10_diff1 40.82184056229221 type value nauc_ndcg_at_10_max 23.832384873845786 type value nauc_ndcg_at_10_std 2.4835478280573966 type value nauc_ndcg_at_1_diff1 49.966125488185206 type value nauc_ndcg_at_1_max 27.193710462071348 type value nauc_ndcg_at_1_std -2.2786990240033718 type value nauc_ndcg_at_20_diff1 40.648257910495396 type value nauc_ndcg_at_20_max 25.1143676738966 type value nauc_ndcg_at_20_std 2.2994895733337084 type value nauc_ndcg_at_3_diff1 42.115026070978224 type value nauc_ndcg_at_3_max 22.895171049309084 type value nauc_ndcg_at_3_std -2.160818780944711 type value nauc_ndcg_at_5_diff1 41.608274106869516 type value nauc_ndcg_at_5_max 23.8694881434902 type value nauc_ndcg_at_5_std -0.2034244843217431 type value nauc_precision_at_1000_diff1 -0.08291845059826138 type value nauc_precision_at_1000_max 20.313650012376964 type value nauc_precision_at_1000_std 13.510706405842074 type value nauc_precision_at_100_diff1 9.885311318637227 type value nauc_precision_at_100_max 26.374081882816075 type value nauc_precision_at_100_std 12.021731392392521 type value nauc_precision_at_10_diff1 25.883633917220507 type value nauc_precision_at_10_max 26.552638392568888 type value nauc_precision_at_10_std 14.460458912586468 type value nauc_precision_at_1_diff1 49.966125488185206 type value nauc_precision_at_1_max 27.193710462071348 type value nauc_precision_at_1_std -2.2786990240033718 type value nauc_precision_at_20_diff1 20.695053025711932 type value nauc_precision_at_20_max 29.151449538281586 type value nauc_precision_at_20_std 13.496486151691874 type value nauc_precision_at_3_diff1 33.475423305252995 type value nauc_precision_at_3_max 24.486060318210537 type value nauc_precision_at_3_std 1.9847009660547001 type value nauc_precision_at_5_diff1 31.14043721035368 type value nauc_precision_at_5_max 27.224889907879906 type value nauc_precision_at_5_std 6.539905565691817 type value nauc_recall_at_1000_diff1 34.33506268392135 type value nauc_recall_at_1000_max 37.11939420491589 type value nauc_recall_at_1000_std 31.371417780064085 type value nauc_recall_at_100_diff1 26.348832193119886 type value nauc_recall_at_100_max 28.096364816659065 type value nauc_recall_at_100_std 11.980597075104523 type value nauc_recall_at_10_diff1 31.684763745718985 type value nauc_recall_at_10_max 21.556273820201323 type value nauc_recall_at_10_std 10.480665669920347 type value nauc_recall_at_1_diff1 50.922871467903654 type value nauc_recall_at_1_max 24.6067671408884 type value nauc_recall_at_1_std -4.630126214452492 type value nauc_recall_at_20_diff1 30.160960913064304 type value nauc_recall_at_20_max 26.303437539000505 type value nauc_recall_at_20_std 10.389326804314718 type value nauc_recall_at_3_diff1 36.88184391262179 type value nauc_recall_at_3_max 20.190953608016223 type value nauc_recall_at_3_std -1.3089868832214695 type value nauc_recall_at_5_diff1 34.99254305849935 type value nauc_recall_at_5_max 22.230820355560727 type value nauc_recall_at_5_std 2.678023175693563 type value ndcg_at_1 27.119 type value ndcg_at_10 39.971000000000004 type value ndcg_at_100 44.952 type value ndcg_at_1000 46.821 type value ndcg_at_20 41.881 type value ndcg_at_3 34.727000000000004 type value ndcg_at_5 36.814 type value precision_at_1 27.119 type value precision_at_10 6.271 type value precision_at_100 0.9249999999999999 type value precision_at_1000 0.11199999999999999 type value precision_at_20 3.605 type value precision_at_3 14.991 type value precision_at_5 10.26 type value recall_at_1 25.06 type value recall_at_10 54.635 type value recall_at_100 77.639 type value recall_at_1000 91.301 type value recall_at_20 61.763 type value recall_at_3 40.143 type value recall_at_5 45.193 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackMathematicaRetrieval 90fceea13679c63fe563ded68f3b6f06e50061de test mteb/cqadupstack-mathematica type value main_score 30.308 type value map_at_1 16.154 type value map_at_10 24.743000000000002 type value map_at_100 26.069 type value map_at_1000 26.197 type value map_at_20 25.46 type value map_at_3 21.816 type value map_at_5 23.443 type value mrr_at_1 20.149253731343283 type value mrr_at_10 29.10847547974411 type value mrr_at_100 30.13595361660887 type value mrr_at_1000 30.211025784243766 type value mrr_at_20 29.706267830545784 type value mrr_at_3 26.451077943615264 type value mrr_at_5 27.868988391376444 type value nauc_map_at_1000_diff1 31.47263576493308 type value nauc_map_at_1000_max 18.49384617286511 type value nauc_map_at_1000_std 0.5754985941500461 type value nauc_map_at_100_diff1 31.44160594144755 type value nauc_map_at_100_max 18.46607563648124 type value nauc_map_at_100_std 0.5879794819886102 type value nauc_map_at_10_diff1 31.71626861875994 type value nauc_map_at_10_max 18.662179744257916 type value nauc_map_at_10_std -0.013163124651967131 type value nauc_map_at_1_diff1 37.33971420967126 type value nauc_map_at_1_max 17.543923177907566 type value nauc_map_at_1_std -0.6312070176608349 type value nauc_map_at_20_diff1 31.443960381506987 type value nauc_map_at_20_max 18.39695256653282 type value nauc_map_at_20_std 0.24204111048796523 type value nauc_map_at_3_diff1 32.66647821102399 type value nauc_map_at_3_max 17.166769100670678 type value nauc_map_at_3_std 0.2511302116485242 type value nauc_map_at_5_diff1 31.814363889022516 type value nauc_map_at_5_max 17.450292361372707 type value nauc_map_at_5_std -0.45123652210324744 type value nauc_mrr_at_1000_diff1 31.885514197021163 type value nauc_mrr_at_1000_max 18.697001653609462 type value nauc_mrr_at_1000_std -0.7075589181761113 type value nauc_mrr_at_100_diff1 31.859235999194958 type value nauc_mrr_at_100_max 18.685923862530778 type value nauc_mrr_at_100_std -0.7027394321332194 type value nauc_mrr_at_10_diff1 32.00819090481358 type value nauc_mrr_at_10_max 18.858552402155677 type value nauc_mrr_at_10_std -0.8729017160389365 type value nauc_mrr_at_1_diff1 36.55463496530352 type value nauc_mrr_at_1_max 17.893580417517832 type value nauc_mrr_at_1_std -2.7268036629932895 type value nauc_mrr_at_20_diff1 31.79086317678036 type value nauc_mrr_at_20_max 18.72847970596078 type value nauc_mrr_at_20_std -0.7526268512949703 type value nauc_mrr_at_3_diff1 32.24844813811655 type value nauc_mrr_at_3_max 17.810304497390504 type value nauc_mrr_at_3_std -1.3573591649881485 type value nauc_mrr_at_5_diff1 32.29719658849603 type value nauc_mrr_at_5_max 18.01176246232617 type value nauc_mrr_at_5_std -1.3156140758149915 type value nauc_ndcg_at_1000_diff1 30.420235654700672 type value nauc_ndcg_at_1000_max 20.14284394608303 type value nauc_ndcg_at_1000_std 2.409633449702056 type value nauc_ndcg_at_100_diff1 29.54867297316048 type value nauc_ndcg_at_100_max 19.63470407851956 type value nauc_ndcg_at_100_std 3.062730904774899 type value nauc_ndcg_at_10_diff1 30.288655944213627 type value nauc_ndcg_at_10_max 20.304033843092395 type value nauc_ndcg_at_10_std 0.7042902099149692 type value nauc_ndcg_at_1_diff1 36.55463496530352 type value nauc_ndcg_at_1_max 17.893580417517832 type value nauc_ndcg_at_1_std -2.7268036629932895 type value nauc_ndcg_at_20_diff1 29.315712836253248 type value nauc_ndcg_at_20_max 19.55539590463071 type value nauc_ndcg_at_20_std 1.4238452417516618 type value nauc_ndcg_at_3_diff1 31.54355638372054 type value nauc_ndcg_at_3_max 17.766299875547816 type value nauc_ndcg_at_3_std 0.28964137714040095 type value nauc_ndcg_at_5_diff1 30.818060499932542 type value nauc_ndcg_at_5_max 18.068091310151164 type value nauc_ndcg_at_5_std -0.16020203299958868 type value nauc_precision_at_1000_diff1 1.8177927649439825 type value nauc_precision_at_1000_max 1.9156412467603505 type value nauc_precision_at_1000_std -1.0195378172264247 type value nauc_precision_at_100_diff1 7.852064632368817 type value nauc_precision_at_100_max 11.41378732164787 type value nauc_precision_at_100_std 8.845589790612463 type value nauc_precision_at_10_diff1 19.576158908850957 type value nauc_precision_at_10_max 22.963840017872794 type value nauc_precision_at_10_std 2.426835326713512 type value nauc_precision_at_1_diff1 36.55463496530352 type value nauc_precision_at_1_max 17.893580417517832 type value nauc_precision_at_1_std -2.7268036629932895 type value nauc_precision_at_20_diff1 15.305985286454149 type value nauc_precision_at_20_max 18.827005672571858 type value nauc_precision_at_20_std 3.992229421735929 type value nauc_precision_at_3_diff1 26.358279542321966 type value nauc_precision_at_3_max 19.340749761958552 type value nauc_precision_at_3_std 0.8501109386129221 type value nauc_precision_at_5_diff1 22.462129435924727 type value nauc_precision_at_5_max 18.890119720243188 type value nauc_precision_at_5_std 0.21756962337473482 type value nauc_recall_at_1000_diff1 25.079504569576184 type value nauc_recall_at_1000_max 36.71138367024086 type value nauc_recall_at_1000_std 18.882277140819067 type value nauc_recall_at_100_diff1 19.980741195591563 type value nauc_recall_at_100_max 21.648381374802273 type value nauc_recall_at_100_std 14.541121099803092 type value nauc_recall_at_10_diff1 24.61930855038573 type value nauc_recall_at_10_max 22.98083391642699 type value nauc_recall_at_10_std 2.860945348018573 type value nauc_recall_at_1_diff1 37.33971420967126 type value nauc_recall_at_1_max 17.543923177907566 type value nauc_recall_at_1_std -0.6312070176608349 type value nauc_recall_at_20_diff1 20.478434900407255 type value nauc_recall_at_20_max 20.439655780702832 type value nauc_recall_at_20_std 5.4039574030039885 type value nauc_recall_at_3_diff1 27.845972047264578 type value nauc_recall_at_3_max 16.649682003649193 type value nauc_recall_at_3_std 2.171037068117454 type value nauc_recall_at_5_diff1 26.76354795664187 type value nauc_recall_at_5_max 17.488511178851763 type value nauc_recall_at_5_std 0.7909085800561211 type value ndcg_at_1 20.149 type value ndcg_at_10 30.308 type value ndcg_at_100 36.361 type value ndcg_at_1000 39.128 type value ndcg_at_20 32.719 type value ndcg_at_3 24.969 type value ndcg_at_5 27.409 type value precision_at_1 20.149 type value precision_at_10 5.784000000000001 type value precision_at_100 1.011 type value precision_at_1000 0.13799999999999998 type value precision_at_20 3.5319999999999996 type value precision_at_3 12.106 type value precision_at_5 9.030000000000001 type value recall_at_1 16.154 type value recall_at_10 43.092000000000006 type value recall_at_100 68.998 type value recall_at_1000 88.127 type value recall_at_20 51.937999999999995 type value recall_at_3 28.473 type value recall_at_5 34.624 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackPhysicsRetrieval 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4 test mteb/cqadupstack-physics type value main_score 46.931 type value map_at_1 30.036 type value map_at_10 40.753 type value map_at_100 42.098 type value map_at_1000 42.201 type value map_at_20 41.494 type value map_at_3 37.55 type value map_at_5 39.266 type value mrr_at_1 36.57362848893166 type value mrr_at_10 46.15953985058891 type value mrr_at_100 46.964409847048735 type value mrr_at_1000 47.006684152310186 type value mrr_at_20 46.63576095668375 type value mrr_at_3 43.39108116778952 type value mrr_at_5 45.0609560474815 type value nauc_map_at_1000_diff1 50.008393482865934 type value nauc_map_at_1000_max 27.44292854668337 type value nauc_map_at_1000_std -1.1827744848485413 type value nauc_map_at_100_diff1 50.01593736030433 type value nauc_map_at_100_max 27.401227555060693 type value nauc_map_at_100_std -1.226830892874052 type value nauc_map_at_10_diff1 50.22186852707843 type value nauc_map_at_10_max 26.882005386152162 type value nauc_map_at_10_std -1.7817280491798217 type value nauc_map_at_1_diff1 53.70420852974904 type value nauc_map_at_1_max 25.134260139256465 type value nauc_map_at_1_std -5.16360510676616 type value nauc_map_at_20_diff1 50.03553131371993 type value nauc_map_at_20_max 27.028712351429306 type value nauc_map_at_20_std -1.4264982725018232 type value nauc_map_at_3_diff1 50.56170061459129 type value nauc_map_at_3_max 27.125222360081885 type value nauc_map_at_3_std -2.1772011676637457 type value nauc_map_at_5_diff1 50.55287654401218 type value nauc_map_at_5_max 27.179943148291034 type value nauc_map_at_5_std -1.9278191493666326 type value nauc_mrr_at_1000_diff1 50.19001608358556 type value nauc_mrr_at_1000_max 30.11015154646845 type value nauc_mrr_at_1000_std -0.01731538046574592 type value nauc_mrr_at_100_diff1 50.17990723644671 type value nauc_mrr_at_100_max 30.08888004508371 type value nauc_mrr_at_100_std -0.03777479539357456 type value nauc_mrr_at_10_diff1 50.29875316793952 type value nauc_mrr_at_10_max 30.0700394599554 type value nauc_mrr_at_10_std -0.1129279328368799 type value nauc_mrr_at_1_diff1 53.13267349109123 type value nauc_mrr_at_1_max 29.600631965679142 type value nauc_mrr_at_1_std -1.0534342020289145 type value nauc_mrr_at_20_diff1 50.20426738346865 type value nauc_mrr_at_20_max 30.03033165917099 type value nauc_mrr_at_20_std -0.0990630706915973 type value nauc_mrr_at_3_diff1 50.44930547118647 type value nauc_mrr_at_3_max 30.18069271699821 type value nauc_mrr_at_3_std -0.4106548753200651 type value nauc_mrr_at_5_diff1 50.42405239937933 type value nauc_mrr_at_5_max 30.323511080797132 type value nauc_mrr_at_5_std -0.10914898852912731 type value nauc_ndcg_at_1000_diff1 48.4023648301636 type value nauc_ndcg_at_1000_max 29.372043713546457 type value nauc_ndcg_at_1000_std 1.4160068477128542 type value nauc_ndcg_at_100_diff1 48.43331450594402 type value nauc_ndcg_at_100_max 28.62936981969224 type value nauc_ndcg_at_100_std 0.8983763064461433 type value nauc_ndcg_at_10_diff1 49.03974183137114 type value nauc_ndcg_at_10_max 27.134352966349006 type value nauc_ndcg_at_10_std -0.7394110214476277 type value nauc_ndcg_at_1_diff1 53.13267349109123 type value nauc_ndcg_at_1_max 29.600631965679142 type value nauc_ndcg_at_1_std -1.0534342020289145 type value nauc_ndcg_at_20_diff1 48.48145045039039 type value nauc_ndcg_at_20_max 27.312478220117836 type value nauc_ndcg_at_20_std -0.08007639532022988 type value nauc_ndcg_at_3_diff1 49.795198984753725 type value nauc_ndcg_at_3_max 28.851373164423457 type value nauc_ndcg_at_3_std -0.7114306314589505 type value nauc_ndcg_at_5_diff1 49.76549299850904 type value nauc_ndcg_at_5_max 28.333095297025384 type value nauc_ndcg_at_5_std -0.6065340225903514 type value nauc_precision_at_1000_diff1 -14.995860825405593 type value nauc_precision_at_1000_max 10.497503977177239 type value nauc_precision_at_1000_std 15.472908805216562 type value nauc_precision_at_100_diff1 -5.056728888436733 type value nauc_precision_at_100_max 16.225279572994932 type value nauc_precision_at_100_std 17.333024162674036 type value nauc_precision_at_10_diff1 18.485355184593836 type value nauc_precision_at_10_max 21.53388484848657 type value nauc_precision_at_10_std 9.864926100512946 type value nauc_precision_at_1_diff1 53.13267349109123 type value nauc_precision_at_1_max 29.600631965679142 type value nauc_precision_at_1_std -1.0534342020289145 type value nauc_precision_at_20_diff1 9.420119338006966 type value nauc_precision_at_20_max 19.132214665647382 type value nauc_precision_at_20_std 13.168229893698857 type value nauc_precision_at_3_diff1 34.51509644827664 type value nauc_precision_at_3_max 28.988501800675305 type value nauc_precision_at_3_std 6.887544108087535 type value nauc_precision_at_5_diff1 29.728890633704864 type value nauc_precision_at_5_max 27.527807375891044 type value nauc_precision_at_5_std 8.615115789487243 type value nauc_recall_at_1000_diff1 21.30536250453658 type value nauc_recall_at_1000_max 45.66826079811565 type value nauc_recall_at_1000_std 40.948489257124734 type value nauc_recall_at_100_diff1 36.41578755512283 type value nauc_recall_at_100_max 25.843843547872236 type value nauc_recall_at_100_std 9.98566808528975 type value nauc_recall_at_10_diff1 42.73428373449279 type value nauc_recall_at_10_max 22.45723124505396 type value nauc_recall_at_10_std 0.6596133636511106 type value nauc_recall_at_1_diff1 53.70420852974904 type value nauc_recall_at_1_max 25.134260139256465 type value nauc_recall_at_1_std -5.16360510676616 type value nauc_recall_at_20_diff1 39.67103657607903 type value nauc_recall_at_20_max 21.767425036370714 type value nauc_recall_at_20_std 2.792759310018829 type value nauc_recall_at_3_diff1 46.672591160111224 type value nauc_recall_at_3_max 26.876529270231792 type value nauc_recall_at_3_std -1.1160005181114536 type value nauc_recall_at_5_diff1 45.77174277314153 type value nauc_recall_at_5_max 26.349199537996853 type value nauc_recall_at_5_std -0.09454430813945205 type value ndcg_at_1 36.574 type value ndcg_at_10 46.931 type value ndcg_at_100 52.40899999999999 type value ndcg_at_1000 54.31 type value ndcg_at_20 49.098000000000006 type value ndcg_at_3 41.585 type value ndcg_at_5 44.009 type value precision_at_1 36.574 type value precision_at_10 8.518 type value precision_at_100 1.317 type value precision_at_1000 0.167 type value precision_at_20 4.99 type value precision_at_3 19.794999999999998 type value precision_at_5 13.879 type value recall_at_1 30.036 type value recall_at_10 60.043 type value recall_at_100 82.78999999999999 type value recall_at_1000 95.017 type value recall_at_20 67.509 type value recall_at_3 44.769 type value recall_at_5 51.23 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackProgrammersRetrieval 6184bc1440d2dbc7612be22b50686b8826d22b32 test mteb/cqadupstack-programmers type value main_score 43.147999999999996 type value map_at_1 27.299 type value map_at_10 37.441 type value map_at_100 38.977000000000004 type value map_at_1000 39.068999999999996 type value map_at_20 38.282 type value map_at_3 34.217 type value map_at_5 36.027 type value mrr_at_1 33.44748858447489 type value mrr_at_10 42.456738783793554 type value mrr_at_100 43.485313174917046 type value mrr_at_1000 43.52577210412886 type value mrr_at_20 43.02431629929082 type value mrr_at_3 39.72602739726027 type value mrr_at_5 41.32420091324198 type value nauc_map_at_1000_diff1 42.430993099089214 type value nauc_map_at_1000_max 28.098034312926952 type value nauc_map_at_1000_std 3.231295090968473 type value nauc_map_at_100_diff1 42.42649976590143 type value nauc_map_at_100_max 28.07518501114065 type value nauc_map_at_100_std 3.2663627223954257 type value nauc_map_at_10_diff1 42.37108247761657 type value nauc_map_at_10_max 27.784006301694887 type value nauc_map_at_10_std 2.1562734801370382 type value nauc_map_at_1_diff1 46.996543750833226 type value nauc_map_at_1_max 23.22775877678291 type value nauc_map_at_1_std -3.185987618625673 type value nauc_map_at_20_diff1 42.285605547136605 type value nauc_map_at_20_max 27.87619604505037 type value nauc_map_at_20_std 2.868182127790041 type value nauc_map_at_3_diff1 43.17884748984982 type value nauc_map_at_3_max 26.640107029543174 type value nauc_map_at_3_std -0.6337177522670645 type value nauc_map_at_5_diff1 42.55295619170691 type value nauc_map_at_5_max 27.09386543850697 type value nauc_map_at_5_std 1.1172301120800785 type value nauc_mrr_at_1000_diff1 41.44240071604904 type value nauc_mrr_at_1000_max 29.942727017459177 type value nauc_mrr_at_1000_std 4.847580130462551 type value nauc_mrr_at_100_diff1 41.43634208329461 type value nauc_mrr_at_100_max 29.94502158371524 type value nauc_mrr_at_100_std 4.873085525046516 type value nauc_mrr_at_10_diff1 41.434406767394215 type value nauc_mrr_at_10_max 29.961051443508534 type value nauc_mrr_at_10_std 4.490183376727645 type value nauc_mrr_at_1_diff1 46.01681006012476 type value nauc_mrr_at_1_max 28.39735171499139 type value nauc_mrr_at_1_std 0.8500045602957598 type value nauc_mrr_at_20_diff1 41.324947979964605 type value nauc_mrr_at_20_max 29.939799023317963 type value nauc_mrr_at_20_std 4.8458435024129685 type value nauc_mrr_at_3_diff1 41.87918200877444 type value nauc_mrr_at_3_max 29.878707844397507 type value nauc_mrr_at_3_std 2.754394941481161 type value nauc_mrr_at_5_diff1 41.17158211294708 type value nauc_mrr_at_5_max 29.525114418603625 type value nauc_mrr_at_5_std 3.6695976231626792 type value nauc_ndcg_at_1000_diff1 40.85015584223998 type value nauc_ndcg_at_1000_max 30.175833847400003 type value nauc_ndcg_at_1000_std 7.454581754774201 type value nauc_ndcg_at_100_diff1 40.679563549502475 type value nauc_ndcg_at_100_max 30.105638179098303 type value nauc_ndcg_at_100_std 8.61962835140906 type value nauc_ndcg_at_10_diff1 40.37700967457906 type value nauc_ndcg_at_10_max 29.33300077317775 type value nauc_ndcg_at_10_std 5.023758212980035 type value nauc_ndcg_at_1_diff1 46.01681006012476 type value nauc_ndcg_at_1_max 28.39735171499139 type value nauc_ndcg_at_1_std 0.8500045602957598 type value nauc_ndcg_at_20_diff1 39.98886010789604 type value nauc_ndcg_at_20_max 29.36296219371212 type value nauc_ndcg_at_20_std 7.1201782062536925 type value nauc_ndcg_at_3_diff1 40.92324084648135 type value nauc_ndcg_at_3_max 28.520942397787785 type value nauc_ndcg_at_3_std 1.0293165278727892 type value nauc_ndcg_at_5_diff1 40.317533959797814 type value nauc_ndcg_at_5_max 28.339428764903264 type value nauc_ndcg_at_5_std 3.1896497530161687 type value nauc_precision_at_1000_diff1 -6.9969817860247625 type value nauc_precision_at_1000_max 9.347778794059506 type value nauc_precision_at_1000_std 7.9646208472184625 type value nauc_precision_at_100_diff1 2.991937395454712 type value nauc_precision_at_100_max 18.71624281667294 type value nauc_precision_at_100_std 21.600526590609512 type value nauc_precision_at_10_diff1 18.37445514123775 type value nauc_precision_at_10_max 29.699257376065063 type value nauc_precision_at_10_std 18.095751349204832 type value nauc_precision_at_1_diff1 46.01681006012476 type value nauc_precision_at_1_max 28.39735171499139 type value nauc_precision_at_1_std 0.8500045602957598 type value nauc_precision_at_20_diff1 11.472713745988054 type value nauc_precision_at_20_max 25.690985880662325 type value nauc_precision_at_20_std 22.46754877988948 type value nauc_precision_at_3_diff1 29.052028827439607 type value nauc_precision_at_3_max 31.04481903220871 type value nauc_precision_at_3_std 8.208096616199493 type value nauc_precision_at_5_diff1 23.711708272374533 type value nauc_precision_at_5_max 30.24946804680551 type value nauc_precision_at_5_std 12.681259000978528 type value nauc_recall_at_1000_diff1 16.82259171106293 type value nauc_recall_at_1000_max 42.76820203485854 type value nauc_recall_at_1000_std 55.97238149176407 type value nauc_recall_at_100_diff1 27.21094062723115 type value nauc_recall_at_100_max 33.698956290459584 type value nauc_recall_at_100_std 37.63664733891902 type value nauc_recall_at_10_diff1 33.26348363515544 type value nauc_recall_at_10_max 29.5718227632449 type value nauc_recall_at_10_std 10.62584355073482 type value nauc_recall_at_1_diff1 46.996543750833226 type value nauc_recall_at_1_max 23.22775877678291 type value nauc_recall_at_1_std -3.185987618625673 type value nauc_recall_at_20_diff1 30.615386537256107 type value nauc_recall_at_20_max 29.459243404458636 type value nauc_recall_at_20_std 18.849849153868913 type value nauc_recall_at_3_diff1 37.22492629427872 type value nauc_recall_at_3_max 27.49351222866847 type value nauc_recall_at_3_std 0.31700586087567145 type value nauc_recall_at_5_diff1 34.4555753891359 type value nauc_recall_at_5_max 27.219221048995283 type value nauc_recall_at_5_std 6.057763073329902 type value ndcg_at_1 33.446999999999996 type value ndcg_at_10 43.147999999999996 type value ndcg_at_100 49.601 type value ndcg_at_1000 51.437 type value ndcg_at_20 45.704 type value ndcg_at_3 37.978 type value ndcg_at_5 40.431 type value precision_at_1 33.446999999999996 type value precision_at_10 7.888000000000001 type value precision_at_100 1.298 type value precision_at_1000 0.16199999999999998 type value precision_at_20 4.749 type value precision_at_3 17.922 type value precision_at_5 12.9 type value recall_at_1 27.299 type value recall_at_10 54.92399999999999 type value recall_at_100 82.308 type value recall_at_1000 94.451 type value recall_at_20 63.952 type value recall_at_3 40.788000000000004 type value recall_at_5 47.198 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackRetrieval CQADupstackRetrieval_is_a_combined_dataset test CQADupstackRetrieval_is_a_combined_dataset type value main_score 42.21466666666666 type value ndcg_at_10 42.21466666666666 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackStatsRetrieval 65ac3a16b8e91f9cee4c9828cc7c335575432a2a test mteb/cqadupstack-stats type value main_score 35.535 type value map_at_1 23.082 type value map_at_10 30.991000000000003 type value map_at_100 31.968000000000004 type value map_at_1000 32.07 type value map_at_20 31.535000000000004 type value map_at_3 28.605000000000004 type value map_at_5 30.06 type value mrr_at_1 25.920245398773005 type value mrr_at_10 33.93191888207225 type value mrr_at_100 34.77251067424867 type value mrr_at_1000 34.838890717603476 type value mrr_at_20 34.396659782410225 type value mrr_at_3 31.722903885480587 type value mrr_at_5 33.11860940695298 type value nauc_map_at_1000_diff1 50.959235687536655 type value nauc_map_at_1000_max 30.655083426929526 type value nauc_map_at_1000_std 4.551329335263164 type value nauc_map_at_100_diff1 50.95439619487166 type value nauc_map_at_100_max 30.623042271335667 type value nauc_map_at_100_std 4.553201745695824 type value nauc_map_at_10_diff1 50.67983398647876 type value nauc_map_at_10_max 30.286986966981583 type value nauc_map_at_10_std 3.9148983660544125 type value nauc_map_at_1_diff1 58.20205266764334 type value nauc_map_at_1_max 28.58134257489169 type value nauc_map_at_1_std 0.40198884745343 type value nauc_map_at_20_diff1 50.90472178620438 type value nauc_map_at_20_max 30.563325966498205 type value nauc_map_at_20_std 4.369655492671673 type value nauc_map_at_3_diff1 52.084512866747325 type value nauc_map_at_3_max 29.374244156637356 type value nauc_map_at_3_std 2.0818606642419963 type value nauc_map_at_5_diff1 51.27705609284862 type value nauc_map_at_5_max 30.17700495077409 type value nauc_map_at_5_std 3.2722185125269103 type value nauc_mrr_at_1000_diff1 51.909591752092425 type value nauc_mrr_at_1000_max 33.36453135370183 type value nauc_mrr_at_1000_std 7.404496516950065 type value nauc_mrr_at_100_diff1 51.900856693619126 type value nauc_mrr_at_100_max 33.350334085938364 type value nauc_mrr_at_100_std 7.410015907741515 type value nauc_mrr_at_10_diff1 51.82074175684569 type value nauc_mrr_at_10_max 33.32820656085001 type value nauc_mrr_at_10_std 7.043558257826565 type value nauc_mrr_at_1_diff1 60.46456002140532 type value nauc_mrr_at_1_max 33.31049028455304 type value nauc_mrr_at_1_std 4.830131026566884 type value nauc_mrr_at_20_diff1 51.8644842944308 type value nauc_mrr_at_20_max 33.3675144190388 type value nauc_mrr_at_20_std 7.256444002173675 type value nauc_mrr_at_3_diff1 52.904828169011154 type value nauc_mrr_at_3_max 32.55024244450511 type value nauc_mrr_at_3_std 6.014060915782276 type value nauc_mrr_at_5_diff1 52.361187623943614 type value nauc_mrr_at_5_max 33.38079408144374 type value nauc_mrr_at_5_std 6.854165091950606 type value nauc_ndcg_at_1000_diff1 48.30949790825087 type value nauc_ndcg_at_1000_max 32.568281800544476 type value nauc_ndcg_at_1000_std 8.966636096573168 type value nauc_ndcg_at_100_diff1 47.9550901718591 type value nauc_ndcg_at_100_max 31.969231434862483 type value nauc_ndcg_at_100_std 8.909343996509326 type value nauc_ndcg_at_10_diff1 47.56929495928323 type value nauc_ndcg_at_10_max 31.131109409439638 type value nauc_ndcg_at_10_std 6.03049937873584 type value nauc_ndcg_at_1_diff1 60.46456002140532 type value nauc_ndcg_at_1_max 33.31049028455304 type value nauc_ndcg_at_1_std 4.830131026566884 type value nauc_ndcg_at_20_diff1 47.99938648902949 type value nauc_ndcg_at_20_max 31.584023047520475 type value nauc_ndcg_at_20_std 7.3552147944361685 type value nauc_ndcg_at_3_diff1 50.28269131499986 type value nauc_ndcg_at_3_max 30.233582570806007 type value nauc_ndcg_at_3_std 3.78476869218036 type value nauc_ndcg_at_5_diff1 49.049921852112895 type value nauc_ndcg_at_5_max 31.174764383636816 type value nauc_ndcg_at_5_std 4.931908749150788 type value nauc_precision_at_1000_diff1 6.883972818358869 type value nauc_precision_at_1000_max 21.834322765687677 type value nauc_precision_at_1000_std 20.000731976327703 type value nauc_precision_at_100_diff1 19.786688523669632 type value nauc_precision_at_100_max 30.328428959273722 type value nauc_precision_at_100_std 26.147922491368902 type value nauc_precision_at_10_diff1 31.41218497795092 type value nauc_precision_at_10_max 33.95003889463453 type value nauc_precision_at_10_std 19.08301072890509 type value nauc_precision_at_1_diff1 60.46456002140532 type value nauc_precision_at_1_max 33.31049028455304 type value nauc_precision_at_1_std 4.830131026566884 type value nauc_precision_at_20_diff1 30.502712564255486 type value nauc_precision_at_20_max 35.178872501427975 type value nauc_precision_at_20_std 23.358935743161783 type value nauc_precision_at_3_diff1 43.1022211297112 type value nauc_precision_at_3_max 33.93732742672912 type value nauc_precision_at_3_std 10.823942310140167 type value nauc_precision_at_5_diff1 38.63486834833309 type value nauc_precision_at_5_max 36.23894828623807 type value nauc_precision_at_5_std 14.675475211699615 type value nauc_recall_at_1000_diff1 23.04089688983766 type value nauc_recall_at_1000_max 40.167606539321355 type value nauc_recall_at_1000_std 43.02153663005034 type value nauc_recall_at_100_diff1 32.000202612409794 type value nauc_recall_at_100_max 31.12741249551696 type value nauc_recall_at_100_std 24.54365478830203 type value nauc_recall_at_10_diff1 36.14374447048929 type value nauc_recall_at_10_max 29.498316079260555 type value nauc_recall_at_10_std 8.641435315254533 type value nauc_recall_at_1_diff1 58.20205266764334 type value nauc_recall_at_1_max 28.58134257489169 type value nauc_recall_at_1_std 0.40198884745343 type value nauc_recall_at_20_diff1 36.22347557385489 type value nauc_recall_at_20_max 29.750817583764405 type value nauc_recall_at_20_std 13.219998916877149 type value nauc_recall_at_3_diff1 43.42606106046774 type value nauc_recall_at_3_max 27.02370831585066 type value nauc_recall_at_3_std 2.148594878901326 type value nauc_recall_at_5_diff1 40.74252027743906 type value nauc_recall_at_5_max 29.661893704694375 type value nauc_recall_at_5_std 5.8950594952457145 type value ndcg_at_1 25.919999999999998 type value ndcg_at_10 35.535 type value ndcg_at_100 40.316 type value ndcg_at_1000 42.84 type value ndcg_at_20 37.424 type value ndcg_at_3 31.223 type value ndcg_at_5 33.521 type value precision_at_1 25.919999999999998 type value precision_at_10 5.736 type value precision_at_100 0.876 type value precision_at_1000 0.117 type value precision_at_20 3.359 type value precision_at_3 13.804 type value precision_at_5 9.754999999999999 type value recall_at_1 23.082 type value recall_at_10 46.399 type value recall_at_100 68.06 type value recall_at_1000 86.821 type value recall_at_20 53.525 type value recall_at_3 34.871 type value recall_at_5 40.492 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackTexRetrieval 46989137a86843e03a6195de44b09deda022eec7 test mteb/cqadupstack-tex type value main_score 29.707 type value map_at_1 17.159 type value map_at_10 24.869 type value map_at_100 26.021 type value map_at_1000 26.151000000000003 type value map_at_20 25.526 type value map_at_3 22.538 type value map_at_5 23.796999999999997 type value mrr_at_1 20.99105299380592 type value mrr_at_10 28.786336971127096 type value mrr_at_100 29.74490721636805 type value mrr_at_1000 29.823214274100618 type value mrr_at_20 29.363881329195756 type value mrr_at_3 26.531314521679345 type value mrr_at_5 27.7339986235376 type value nauc_map_at_1000_diff1 37.13825685322779 type value nauc_map_at_1000_max 25.949209359787055 type value nauc_map_at_1000_std -0.1789880172036093 type value nauc_map_at_100_diff1 37.13565311027618 type value nauc_map_at_100_max 25.909889375022395 type value nauc_map_at_100_std -0.20169274828783654 type value nauc_map_at_10_diff1 37.412949674073325 type value nauc_map_at_10_max 25.837714322449912 type value nauc_map_at_10_std -0.7989713426808079 type value nauc_map_at_1_diff1 43.66535106611136 type value nauc_map_at_1_max 24.934157845499076 type value nauc_map_at_1_std -2.798761696625911 type value nauc_map_at_20_diff1 37.25188765578179 type value nauc_map_at_20_max 25.887533682661708 type value nauc_map_at_20_std -0.48710070531162597 type value nauc_map_at_3_diff1 38.747478927053876 type value nauc_map_at_3_max 25.551679823476835 type value nauc_map_at_3_std -1.5848393331273871 type value nauc_map_at_5_diff1 38.11902875922142 type value nauc_map_at_5_max 25.84766602647597 type value nauc_map_at_5_std -1.0063039730468788 type value nauc_mrr_at_1000_diff1 35.409856966860396 type value nauc_mrr_at_1000_max 27.86922067595656 type value nauc_mrr_at_1000_std -0.0734512410447464 type value nauc_mrr_at_100_diff1 35.400804471162054 type value nauc_mrr_at_100_max 27.866591373962002 type value nauc_mrr_at_100_std -0.04959722841487173 type value nauc_mrr_at_10_diff1 35.5199909370886 type value nauc_mrr_at_10_max 27.962695735822045 type value nauc_mrr_at_10_std -0.5296125062220955 type value nauc_mrr_at_1_diff1 41.65630429652082 type value nauc_mrr_at_1_max 27.826862844728982 type value nauc_mrr_at_1_std -2.0718644041769205 type value nauc_mrr_at_20_diff1 35.38119545574273 type value nauc_mrr_at_20_max 27.888497220693953 type value nauc_mrr_at_20_std -0.2890434589026467 type value nauc_mrr_at_3_diff1 36.603117913849466 type value nauc_mrr_at_3_max 27.947449591583933 type value nauc_mrr_at_3_std -1.0865714056168478 type value nauc_mrr_at_5_diff1 35.92459791709931 type value nauc_mrr_at_5_max 28.035251623858272 type value nauc_mrr_at_5_std -0.8711878495606741 type value nauc_ndcg_at_1000_diff1 34.06248430947299 type value nauc_ndcg_at_1000_max 26.7997542315953 type value nauc_ndcg_at_1000_std 3.3240363708933742 type value nauc_ndcg_at_100_diff1 33.68748871110203 type value nauc_ndcg_at_100_max 26.362138300414788 type value nauc_ndcg_at_100_std 3.3435049793759717 type value nauc_ndcg_at_10_diff1 34.50272053437263 type value nauc_ndcg_at_10_max 26.41321919372202 type value nauc_ndcg_at_10_std 0.44722981908997034 type value nauc_ndcg_at_1_diff1 41.65630429652082 type value nauc_ndcg_at_1_max 27.826862844728982 type value nauc_ndcg_at_1_std -2.0718644041769205 type value nauc_ndcg_at_20_diff1 34.095928245730065 type value nauc_ndcg_at_20_max 26.278658129351108 type value nauc_ndcg_at_20_std 1.333694029082928 type value nauc_ndcg_at_3_diff1 36.69705632103637 type value nauc_ndcg_at_3_max 26.78968350373072 type value nauc_ndcg_at_3_std -1.0804397591306258 type value nauc_ndcg_at_5_diff1 35.72910772416993 type value nauc_ndcg_at_5_max 26.70057707274289 type value nauc_ndcg_at_5_std -0.13486271460127894 type value nauc_precision_at_1000_diff1 0.05861252770643225 type value nauc_precision_at_1000_max 18.601946335509112 type value nauc_precision_at_1000_std 9.800060286260463 type value nauc_precision_at_100_diff1 7.363883419620025 type value nauc_precision_at_100_max 22.20848267682575 type value nauc_precision_at_100_std 12.714551550333642 type value nauc_precision_at_10_diff1 21.331506854435275 type value nauc_precision_at_10_max 28.684902701505965 type value nauc_precision_at_10_std 3.6550639959191207 type value nauc_precision_at_1_diff1 41.65630429652082 type value nauc_precision_at_1_max 27.826862844728982 type value nauc_precision_at_1_std -2.0718644041769205 type value nauc_precision_at_20_diff1 16.844978902521646 type value nauc_precision_at_20_max 27.441958887770646 type value nauc_precision_at_20_std 6.3826805047558315 type value nauc_precision_at_3_diff1 30.639398097322594 type value nauc_precision_at_3_max 29.939776959172697 type value nauc_precision_at_3_std -0.20286831584574574 type value nauc_precision_at_5_diff1 26.70376825047474 type value nauc_precision_at_5_max 29.60604358978513 type value nauc_precision_at_5_std 1.5809149742471655 type value nauc_recall_at_1000_diff1 17.785715749599042 type value nauc_recall_at_1000_max 23.48376672770539 type value nauc_recall_at_1000_std 30.385000337970858 type value nauc_recall_at_100_diff1 21.05284222570054 type value nauc_recall_at_100_max 21.945063586716614 type value nauc_recall_at_100_std 17.466562038077875 type value nauc_recall_at_10_diff1 26.597231762971674 type value nauc_recall_at_10_max 23.5079436519741 type value nauc_recall_at_10_std 3.263135880492641 type value nauc_recall_at_1_diff1 43.66535106611136 type value nauc_recall_at_1_max 24.934157845499076 type value nauc_recall_at_1_std -2.798761696625911 type value nauc_recall_at_20_diff1 24.832091637143787 type value nauc_recall_at_20_max 22.315764495952237 type value nauc_recall_at_20_std 6.129833251765541 type value nauc_recall_at_3_diff1 32.85408650886733 type value nauc_recall_at_3_max 24.409412121823397 type value nauc_recall_at_3_std 0.04999270761091106 type value nauc_recall_at_5_diff1 30.258414223370007 type value nauc_recall_at_5_max 24.512878195644664 type value nauc_recall_at_5_std 1.849046122226546 type value ndcg_at_1 20.991 type value ndcg_at_10 29.707 type value ndcg_at_100 35.043 type value ndcg_at_1000 38.032 type value ndcg_at_20 31.828 type value ndcg_at_3 25.488 type value ndcg_at_5 27.348 type value precision_at_1 20.991 type value precision_at_10 5.416 type value precision_at_100 0.947 type value precision_at_1000 0.13899999999999998 type value precision_at_20 3.324 type value precision_at_3 12.113 type value precision_at_5 8.734 type value recall_at_1 17.159 type value recall_at_10 40.397 type value recall_at_100 64.139 type value recall_at_1000 85.328 type value recall_at_20 48.193000000000005 type value recall_at_3 28.555999999999997 type value recall_at_5 33.394 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackUnixRetrieval 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53 test mteb/cqadupstack-unix type value main_score 41.831 type value map_at_1 25.889 type value map_at_10 36.131 type value map_at_100 37.277 type value map_at_1000 37.383 type value map_at_20 36.797000000000004 type value map_at_3 33.194 type value map_at_5 34.88 type value mrr_at_1 30.69029850746269 type value mrr_at_10 40.34274312959011 type value mrr_at_100 41.15568315924076 type value mrr_at_1000 41.21534922643823 type value mrr_at_20 40.81612888073637 type value mrr_at_3 37.624378109452735 type value mrr_at_5 39.25217661691539 type value nauc_map_at_1000_diff1 51.0475973810661 type value nauc_map_at_1000_max 38.75825500903846 type value nauc_map_at_1000_std 1.6136905986292485 type value nauc_map_at_100_diff1 51.04820272616417 type value nauc_map_at_100_max 38.74584044282816 type value nauc_map_at_100_std 1.5969607728429231 type value nauc_map_at_10_diff1 50.94166583581915 type value nauc_map_at_10_max 38.37738102486977 type value nauc_map_at_10_std 1.2635889890868346 type value nauc_map_at_1_diff1 59.242331404755774 type value nauc_map_at_1_max 39.02663876284084 type value nauc_map_at_1_std -0.4739614669668662 type value nauc_map_at_20_diff1 50.97455684751073 type value nauc_map_at_20_max 38.57646135094768 type value nauc_map_at_20_std 1.4640361871795349 type value nauc_map_at_3_diff1 51.608034622903176 type value nauc_map_at_3_max 38.433045221071325 type value nauc_map_at_3_std 0.5831392788488381 type value nauc_map_at_5_diff1 50.947880732714445 type value nauc_map_at_5_max 38.60925399151572 type value nauc_map_at_5_std 1.291960076749259 type value nauc_mrr_at_1000_diff1 50.210650177335104 type value nauc_mrr_at_1000_max 37.951469256285804 type value nauc_mrr_at_1000_std 0.7902286837699785 type value nauc_mrr_at_100_diff1 50.20638219267218 type value nauc_mrr_at_100_max 37.9377948931531 type value nauc_mrr_at_100_std 0.774713370156735 type value nauc_mrr_at_10_diff1 49.836111870473935 type value nauc_mrr_at_10_max 37.65348449064669 type value nauc_mrr_at_10_std 0.5231944356104865 type value nauc_mrr_at_1_diff1 57.56522049860187 type value nauc_mrr_at_1_max 39.39798439825698 type value nauc_mrr_at_1_std -0.4516317740083426 type value nauc_mrr_at_20_diff1 50.1006649557446 type value nauc_mrr_at_20_max 37.84223094800734 type value nauc_mrr_at_20_std 0.8086280885894073 type value nauc_mrr_at_3_diff1 50.441725884115996 type value nauc_mrr_at_3_max 37.90807984566849 type value nauc_mrr_at_3_std 0.02550782712808399 type value nauc_mrr_at_5_diff1 49.85802035503023 type value nauc_mrr_at_5_max 38.065589153711116 type value nauc_mrr_at_5_std 0.6274639011716443 type value nauc_ndcg_at_1000_diff1 49.03659827838649 type value nauc_ndcg_at_1000_max 39.132735746113575 type value nauc_ndcg_at_1000_std 3.627422164709519 type value nauc_ndcg_at_100_diff1 49.00264137357818 type value nauc_ndcg_at_100_max 39.01919928439472 type value nauc_ndcg_at_100_std 3.558699165061359 type value nauc_ndcg_at_10_diff1 48.26671791603934 type value nauc_ndcg_at_10_max 37.571416815576114 type value nauc_ndcg_at_10_std 1.9403797342170153 type value nauc_ndcg_at_1_diff1 57.56522049860187 type value nauc_ndcg_at_1_max 39.39798439825698 type value nauc_ndcg_at_1_std -0.4516317740083426 type value nauc_ndcg_at_20_diff1 48.66105608484808 type value nauc_ndcg_at_20_max 38.22139553816886 type value nauc_ndcg_at_20_std 2.8911511133753782 type value nauc_ndcg_at_3_diff1 49.00804557017609 type value nauc_ndcg_at_3_max 37.72179482159779 type value nauc_ndcg_at_3_std 0.8400931058476853 type value nauc_ndcg_at_5_diff1 48.24457268105435 type value nauc_ndcg_at_5_max 38.191301845180604 type value nauc_ndcg_at_5_std 1.9471919379129263 type value nauc_precision_at_1000_diff1 -12.33889190395623 type value nauc_precision_at_1000_max -1.3115353486004107 type value nauc_precision_at_1000_std 2.495795006465732 type value nauc_precision_at_100_diff1 2.1703067538960084 type value nauc_precision_at_100_max 15.898479441971332 type value nauc_precision_at_100_std 7.910076672658263 type value nauc_precision_at_10_diff1 22.20759907514248 type value nauc_precision_at_10_max 25.51471885225117 type value nauc_precision_at_10_std 2.3609262388624512 type value nauc_precision_at_1_diff1 57.56522049860187 type value nauc_precision_at_1_max 39.39798439825698 type value nauc_precision_at_1_std -0.4516317740083426 type value nauc_precision_at_20_diff1 15.94035009026911 type value nauc_precision_at_20_max 23.178150944386744 type value nauc_precision_at_20_std 5.207387751900332 type value nauc_precision_at_3_diff1 34.99396954995648 type value nauc_precision_at_3_max 33.14418980052923 type value nauc_precision_at_3_std 1.660740116435417 type value nauc_precision_at_5_diff1 29.544849162475362 type value nauc_precision_at_5_max 32.150735196144645 type value nauc_precision_at_5_std 3.323068902360027 type value nauc_recall_at_1000_diff1 30.978839058267006 type value nauc_recall_at_1000_max 48.722880061794 type value nauc_recall_at_1000_std 46.28381322993451 type value nauc_recall_at_100_diff1 40.22130846505397 type value nauc_recall_at_100_max 38.28644243336189 type value nauc_recall_at_100_std 15.77321980757386 type value nauc_recall_at_10_diff1 38.9910969333204 type value nauc_recall_at_10_max 32.807008720875984 type value nauc_recall_at_10_std 5.065337152044106 type value nauc_recall_at_1_diff1 59.242331404755774 type value nauc_recall_at_1_max 39.02663876284084 type value nauc_recall_at_1_std -0.4739614669668662 type value nauc_recall_at_20_diff1 40.14875646536079 type value nauc_recall_at_20_max 34.83600129324774 type value nauc_recall_at_20_std 9.01370840232733 type value nauc_recall_at_3_diff1 42.65832338786475 type value nauc_recall_at_3_max 35.56970517818321 type value nauc_recall_at_3_std 1.8050805176967801 type value nauc_recall_at_5_diff1 40.07274624634327 type value nauc_recall_at_5_max 35.74226371272684 type value nauc_recall_at_5_std 4.873290118594757 type value ndcg_at_1 30.69 type value ndcg_at_10 41.831 type value ndcg_at_100 46.966 type value ndcg_at_1000 49.334 type value ndcg_at_20 43.927 type value ndcg_at_3 36.534 type value ndcg_at_5 39.126 type value precision_at_1 30.69 type value precision_at_10 7.1739999999999995 type value precision_at_100 1.095 type value precision_at_1000 0.14200000000000002 type value precision_at_20 4.184 type value precision_at_3 16.853 type value precision_at_5 11.922 type value recall_at_1 25.889 type value recall_at_10 54.962999999999994 type value recall_at_100 77.239 type value recall_at_1000 93.729 type value recall_at_20 62.534 type value recall_at_3 40.336 type value recall_at_5 47.083000000000006 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackWebmastersRetrieval 160c094312a0e1facb97e55eeddb698c0abe3571 test mteb/cqadupstack-webmasters type value main_score 41.695 type value map_at_1 26.296999999999997 type value map_at_10 35.929 type value map_at_100 37.625 type value map_at_1000 37.856 type value map_at_20 36.831 type value map_at_3 33.042 type value map_at_5 34.552 type value mrr_at_1 31.422924901185773 type value mrr_at_10 40.36718112805069 type value mrr_at_100 41.48635728771627 type value mrr_at_1000 41.53760971899895 type value mrr_at_20 41.05566983667548 type value mrr_at_3 38.24110671936759 type value mrr_at_5 39.49604743083004 type value nauc_map_at_1000_diff1 45.88419073101831 type value nauc_map_at_1000_max 32.272696879964606 type value nauc_map_at_1000_std 6.435633876271509 type value nauc_map_at_100_diff1 46.118272363764085 type value nauc_map_at_100_max 32.459168722724094 type value nauc_map_at_100_std 6.292246088710509 type value nauc_map_at_10_diff1 46.603302676569655 type value nauc_map_at_10_max 32.38318941747706 type value nauc_map_at_10_std 4.720511340512196 type value nauc_map_at_1_diff1 53.474193431022286 type value nauc_map_at_1_max 30.096745684269028 type value nauc_map_at_1_std 0.1635051536400562 type value nauc_map_at_20_diff1 46.25945687266626 type value nauc_map_at_20_max 32.47553839186572 type value nauc_map_at_20_std 5.566329221862548 type value nauc_map_at_3_diff1 47.86679192761851 type value nauc_map_at_3_max 31.531646616728803 type value nauc_map_at_3_std 3.1837781149112496 type value nauc_map_at_5_diff1 46.4585625030729 type value nauc_map_at_5_max 32.013423473733624 type value nauc_map_at_5_std 4.403527966937636 type value nauc_mrr_at_1000_diff1 44.168029521898646 type value nauc_mrr_at_1000_max 33.231405944995004 type value nauc_mrr_at_1000_std 8.153326593928266 type value nauc_mrr_at_100_diff1 44.17027683367582 type value nauc_mrr_at_100_max 33.23422175046355 type value nauc_mrr_at_100_std 8.198284732472755 type value nauc_mrr_at_10_diff1 44.2496903067119 type value nauc_mrr_at_10_max 33.055178332856116 type value nauc_mrr_at_10_std 7.831026978775937 type value nauc_mrr_at_1_diff1 48.4273290718694 type value nauc_mrr_at_1_max 31.89937877913926 type value nauc_mrr_at_1_std 3.873149993747884 type value nauc_mrr_at_20_diff1 44.09113284049905 type value nauc_mrr_at_20_max 33.22019452622306 type value nauc_mrr_at_20_std 8.133802855890329 type value nauc_mrr_at_3_diff1 44.86167450862544 type value nauc_mrr_at_3_max 32.98194923216794 type value nauc_mrr_at_3_std 6.9890614678195 type value nauc_mrr_at_5_diff1 43.939080994503634 type value nauc_mrr_at_5_max 33.25648484685068 type value nauc_mrr_at_5_std 7.943963197772268 type value nauc_ndcg_at_1000_diff1 43.42006126140444 type value nauc_ndcg_at_1000_max 32.89416354016926 type value nauc_ndcg_at_1000_std 9.740672987523162 type value nauc_ndcg_at_100_diff1 43.737763705105145 type value nauc_ndcg_at_100_max 33.102019342275725 type value nauc_ndcg_at_100_std 10.354524698232671 type value nauc_ndcg_at_10_diff1 43.574979909615 type value nauc_ndcg_at_10_max 32.22335464466024 type value nauc_ndcg_at_10_std 7.827717817165889 type value nauc_ndcg_at_1_diff1 48.4273290718694 type value nauc_ndcg_at_1_max 31.89937877913926 type value nauc_ndcg_at_1_std 3.873149993747884 type value nauc_ndcg_at_20_diff1 43.135943873988566 type value nauc_ndcg_at_20_max 32.88264995288679 type value nauc_ndcg_at_20_std 9.104351404942863 type value nauc_ndcg_at_3_diff1 45.18739397775064 type value nauc_ndcg_at_3_max 31.580166756620283 type value nauc_ndcg_at_3_std 6.137398763080745 type value nauc_ndcg_at_5_diff1 42.950299500112955 type value nauc_ndcg_at_5_max 32.04130248991469 type value nauc_ndcg_at_5_std 8.322547993875903 type value nauc_precision_at_1000_diff1 -23.129419591612365 type value nauc_precision_at_1000_max -11.41420275910081 type value nauc_precision_at_1000_std 19.146268912764334 type value nauc_precision_at_100_diff1 -12.413671568737618 type value nauc_precision_at_100_max 0.537649304108213 type value nauc_precision_at_100_std 27.325180241816415 type value nauc_precision_at_10_diff1 15.277020606429655 type value nauc_precision_at_10_max 23.51972448360081 type value nauc_precision_at_10_std 19.103862771406927 type value nauc_precision_at_1_diff1 48.4273290718694 type value nauc_precision_at_1_max 31.89937877913926 type value nauc_precision_at_1_std 3.873149993747884 type value nauc_precision_at_20_diff1 4.910626579631313 type value nauc_precision_at_20_max 17.000613397246163 type value nauc_precision_at_20_std 24.370825263718903 type value nauc_precision_at_3_diff1 31.259123635562613 type value nauc_precision_at_3_max 28.91653697836493 type value nauc_precision_at_3_std 11.718828024267332 type value nauc_precision_at_5_diff1 21.896001023343413 type value nauc_precision_at_5_max 26.53717311029016 type value nauc_precision_at_5_std 17.506215861477873 type value nauc_recall_at_1000_diff1 15.545423862859614 type value nauc_recall_at_1000_max 33.54097556941026 type value nauc_recall_at_1000_std 41.970927423554926 type value nauc_recall_at_100_diff1 32.29112323650048 type value nauc_recall_at_100_max 31.72353031716839 type value nauc_recall_at_100_std 30.06509939448423 type value nauc_recall_at_10_diff1 36.223842357407875 type value nauc_recall_at_10_max 29.16462133003001 type value nauc_recall_at_10_std 9.404694229411104 type value nauc_recall_at_1_diff1 53.474193431022286 type value nauc_recall_at_1_max 30.096745684269028 type value nauc_recall_at_1_std 0.1635051536400562 type value nauc_recall_at_20_diff1 32.2732032299642 type value nauc_recall_at_20_max 30.699505625402928 type value nauc_recall_at_20_std 15.947782026021681 type value nauc_recall_at_3_diff1 41.746081012759426 type value nauc_recall_at_3_max 29.019436574100016 type value nauc_recall_at_3_std 4.757836484193213 type value nauc_recall_at_5_diff1 35.74337633697516 type value nauc_recall_at_5_max 30.17283125351457 type value nauc_recall_at_5_std 9.488723875013253 type value ndcg_at_1 31.423000000000002 type value ndcg_at_10 41.695 type value ndcg_at_100 48.109 type value ndcg_at_1000 50.39900000000001 type value ndcg_at_20 44.208999999999996 type value ndcg_at_3 37.241 type value ndcg_at_5 39.228 type value precision_at_1 31.423000000000002 type value precision_at_10 7.866 type value precision_at_100 1.603 type value precision_at_1000 0.245 type value precision_at_20 4.99 type value precision_at_3 17.523 type value precision_at_5 12.49 type value recall_at_1 26.296999999999997 type value recall_at_10 52.778000000000006 type value recall_at_100 80.961 type value recall_at_1000 94.894 type value recall_at_20 62.239 type value recall_at_3 39.814 type value recall_at_5 45.381 type Retrieval dataset metrics task config name revision split type default MTEB CQADupstackWordpressRetrieval 4ffe81d471b1924886b33c7567bfb200e9eec5c4 test mteb/cqadupstack-wordpress type value main_score 33.936 type value map_at_1 21.297 type value map_at_10 29.29 type value map_at_100 30.407 type value map_at_1000 30.514999999999997 type value map_at_20 29.983999999999998 type value map_at_3 26.950000000000003 type value map_at_5 28.287000000000003 type value mrr_at_1 23.65988909426987 type value mrr_at_10 31.57996655224012 type value mrr_at_100 32.58133076268842 type value mrr_at_1000 32.659811204298656 type value mrr_at_20 32.18205959735665 type value mrr_at_3 29.482439926062852 type value mrr_at_5 30.600739371534207 type value nauc_map_at_1000_diff1 33.65655465193916 type value nauc_map_at_1000_max 29.523610574712706 type value nauc_map_at_1000_std -0.48883917163984836 type value nauc_map_at_100_diff1 33.657822812150975 type value nauc_map_at_100_max 29.531870292234302 type value nauc_map_at_100_std -0.49454342105691873 type value nauc_map_at_10_diff1 34.03649741206849 type value nauc_map_at_10_max 29.48133710519135 type value nauc_map_at_10_std -1.3003031064360702 type value nauc_map_at_1_diff1 41.319491034458395 type value nauc_map_at_1_max 30.08436727224079 type value nauc_map_at_1_std -4.283931261517225 type value nauc_map_at_20_diff1 33.644132189750856 type value nauc_map_at_20_max 29.57915168728321 type value nauc_map_at_20_std -0.71252104365507 type value nauc_map_at_3_diff1 33.8965524645013 type value nauc_map_at_3_max 28.898722773976697 type value nauc_map_at_3_std -1.8649217196078969 type value nauc_map_at_5_diff1 33.65177546877711 type value nauc_map_at_5_max 29.449552621308055 type value nauc_map_at_5_std -1.9217932476234898 type value nauc_mrr_at_1000_diff1 34.21675867856096 type value nauc_mrr_at_1000_max 30.198504997318466 type value nauc_mrr_at_1000_std 0.5352461648974925 type value nauc_mrr_at_100_diff1 34.210091539379874 type value nauc_mrr_at_100_max 30.19136090320817 type value nauc_mrr_at_100_std 0.5431068443349623 type value nauc_mrr_at_10_diff1 34.50092238629405 type value nauc_mrr_at_10_max 30.360381404088816 type value nauc_mrr_at_10_std 0.007947172236616928 type value nauc_mrr_at_1_diff1 41.47500594264137 type value nauc_mrr_at_1_max 30.932862195893563 type value nauc_mrr_at_1_std -3.0060183101242157 type value nauc_mrr_at_20_diff1 34.15523281231642 type value nauc_mrr_at_20_max 30.251528444714324 type value nauc_mrr_at_20_std 0.41483749048122587 type value nauc_mrr_at_3_diff1 34.54541333351149 type value nauc_mrr_at_3_max 30.357741809442512 type value nauc_mrr_at_3_std -0.5977586679572796 type value nauc_mrr_at_5_diff1 34.058033979119465 type value nauc_mrr_at_5_max 30.19093785155445 type value nauc_mrr_at_5_std -0.6829700596355942 type value nauc_ndcg_at_1000_diff1 31.530363860261506 type value nauc_ndcg_at_1000_max 29.90327018263153 type value nauc_ndcg_at_1000_std 3.033100623143071 type value nauc_ndcg_at_100_diff1 31.56967408174602 type value nauc_ndcg_at_100_max 29.53643288504651 type value nauc_ndcg_at_100_std 3.4997411689634883 type value nauc_ndcg_at_10_diff1 32.27374955735248 type value nauc_ndcg_at_10_max 29.519348153684348 type value nauc_ndcg_at_10_std 0.3042011208954651 type value nauc_ndcg_at_1_diff1 41.47500594264137 type value nauc_ndcg_at_1_max 30.932862195893563 type value nauc_ndcg_at_1_std -3.0060183101242157 type value nauc_ndcg_at_20_diff1 31.102403306150194 type value nauc_ndcg_at_20_max 29.677553740846967 type value nauc_ndcg_at_20_std 2.1195261321395766 type value nauc_ndcg_at_3_diff1 32.02333047452249 type value nauc_ndcg_at_3_max 28.888372073027796 type value nauc_ndcg_at_3_std -0.924661397180436 type value nauc_ndcg_at_5_diff1 31.466174122311667 type value nauc_ndcg_at_5_max 29.307628068867754 type value nauc_ndcg_at_5_std -1.2046829876982417 type value nauc_precision_at_1000_diff1 -6.075546300902165 type value nauc_precision_at_1000_max -2.187623217222419 type value nauc_precision_at_1000_std 12.584752959282211 type value nauc_precision_at_100_diff1 14.295101079499434 type value nauc_precision_at_100_max 20.388641516894513 type value nauc_precision_at_100_std 21.887960759975524 type value nauc_precision_at_10_diff1 24.536039003837043 type value nauc_precision_at_10_max 29.357326635020637 type value nauc_precision_at_10_std 7.65955284021577 type value nauc_precision_at_1_diff1 41.47500594264137 type value nauc_precision_at_1_max 30.932862195893563 type value nauc_precision_at_1_std -3.0060183101242157 type value nauc_precision_at_20_diff1 18.634308701475955 type value nauc_precision_at_20_max 27.88621903726711 type value nauc_precision_at_20_std 14.96789816785273 type value nauc_precision_at_3_diff1 26.928594601514146 type value nauc_precision_at_3_max 29.653482500006007 type value nauc_precision_at_3_std 2.114869053308719 type value nauc_precision_at_5_diff1 24.137817643228992 type value nauc_precision_at_5_max 29.467809315433215 type value nauc_precision_at_5_std 2.2335268351775777 type value nauc_recall_at_1000_diff1 7.561889223723366 type value nauc_recall_at_1000_max 34.64462683484328 type value nauc_recall_at_1000_std 32.07766726976165 type value nauc_recall_at_100_diff1 21.87202458692393 type value nauc_recall_at_100_max 26.060326662408357 type value nauc_recall_at_100_std 20.038540279921996 type value nauc_recall_at_10_diff1 26.59257905849799 type value nauc_recall_at_10_max 27.840231433969887 type value nauc_recall_at_10_std 3.547350776489353 type value nauc_recall_at_1_diff1 41.319491034458395 type value nauc_recall_at_1_max 30.08436727224079 type value nauc_recall_at_1_std -4.283931261517225 type value nauc_recall_at_20_diff1 21.84062118775981 type value nauc_recall_at_20_max 27.960813344120865 type value nauc_recall_at_20_std 9.945117730379264 type value nauc_recall_at_3_diff1 26.240584213234957 type value nauc_recall_at_3_max 27.32563942378109 type value nauc_recall_at_3_std 0.15754039149189397 type value nauc_recall_at_5_diff1 25.327116061029542 type value nauc_recall_at_5_max 28.12294625143933 type value nauc_recall_at_5_std -0.7151467503960333 type value ndcg_at_1 23.66 type value ndcg_at_10 33.936 type value ndcg_at_100 39.172000000000004 type value ndcg_at_1000 41.858000000000004 type value ndcg_at_20 36.248999999999995 type value ndcg_at_3 29.454 type value ndcg_at_5 31.555 type value precision_at_1 23.66 type value precision_at_10 5.323 type value precision_at_100 0.8500000000000001 type value precision_at_1000 0.11900000000000001 type value precision_at_20 3.216 type value precision_at_3 12.568999999999999 type value precision_at_5 8.909 type value recall_at_1 21.297 type value recall_at_10 46.007 type value recall_at_100 69.73700000000001 type value recall_at_1000 89.91900000000001 type value recall_at_20 54.806 type value recall_at_3 33.727000000000004 type value recall_at_5 38.675 type Retrieval dataset metrics task config name revision split type default MTEB ClimateFEVER 47f2ac6acb640fc46020b02a5b59fdda04d39380 test mteb/climate-fever type value main_score 27.794999999999998 type value map_at_1 11.138 type value map_at_10 19.56 type value map_at_100 21.416 type value map_at_1000 21.6 type value map_at_20 20.556 type value map_at_3 16.066 type value map_at_5 17.883 type value mrr_at_1 24.364820846905538 type value mrr_at_10 36.21314823432085 type value mrr_at_100 37.17398469553677 type value mrr_at_1000 37.21013480329614 type value mrr_at_20 36.78840955357927 type value mrr_at_3 32.486427795874 type value mrr_at_5 34.77958740499451 type value nauc_map_at_1000_diff1 22.20775473369687 type value nauc_map_at_1000_max 36.19769366030157 type value nauc_map_at_1000_std 17.568432565671753 type value nauc_map_at_100_diff1 22.202037755951228 type value nauc_map_at_100_max 36.13800341266643 type value nauc_map_at_100_std 17.486248132972992 type value nauc_map_at_10_diff1 22.9042018284273 type value nauc_map_at_10_max 36.08475064127247 type value nauc_map_at_10_std 15.726587888083884 type value nauc_map_at_1_diff1 28.652249616122717 type value nauc_map_at_1_max 32.05131359795648 type value nauc_map_at_1_std 11.262948253532807 type value nauc_map_at_20_diff1 22.451026108322598 type value nauc_map_at_20_max 36.32385371085683 type value nauc_map_at_20_std 16.64337500445571 type value nauc_map_at_3_diff1 23.16011840834893 type value nauc_map_at_3_max 33.24586608916762 type value nauc_map_at_3_std 12.56332091941363 type value nauc_map_at_5_diff1 22.93957941358747 type value nauc_map_at_5_max 34.699460514009 type value nauc_map_at_5_std 14.063661191876298 type value nauc_mrr_at_1000_diff1 23.24777062437872 type value nauc_mrr_at_1000_max 33.450026215376866 type value nauc_mrr_at_1000_std 20.426349474081853 type value nauc_mrr_at_100_diff1 23.23401699847253 type value nauc_mrr_at_100_max 33.45459692613422 type value nauc_mrr_at_100_std 20.440070448958714 type value nauc_mrr_at_10_diff1 23.281604083585396 type value nauc_mrr_at_10_max 33.4988527620155 type value nauc_mrr_at_10_std 20.367252947781857 type value nauc_mrr_at_1_diff1 26.355328110953575 type value nauc_mrr_at_1_max 30.471508547730092 type value nauc_mrr_at_1_std 16.11568495246132 type value nauc_mrr_at_20_diff1 23.140683139461732 type value nauc_mrr_at_20_max 33.48554958878313 type value nauc_mrr_at_20_std 20.44494070529154 type value nauc_mrr_at_3_diff1 23.301943271387042 type value nauc_mrr_at_3_max 32.422994068557635 type value nauc_mrr_at_3_std 18.939596173947923 type value nauc_mrr_at_5_diff1 23.33922409006143 type value nauc_mrr_at_5_max 33.0752792306208 type value nauc_mrr_at_5_std 19.768166202806604 type value nauc_ndcg_at_1000_diff1 19.998881742304263 type value nauc_ndcg_at_1000_max 37.786993391629984 type value nauc_ndcg_at_1000_std 24.51994563648888 type value nauc_ndcg_at_100_diff1 19.98463107036392 type value nauc_ndcg_at_100_max 37.00722603001812 type value nauc_ndcg_at_100_std 23.717744758974426 type value nauc_ndcg_at_10_diff1 21.62784861661253 type value nauc_ndcg_at_10_max 37.16285223589196 type value nauc_ndcg_at_10_std 19.34332938831155 type value nauc_ndcg_at_1_diff1 26.355328110953575 type value nauc_ndcg_at_1_max 30.471508547730092 type value nauc_ndcg_at_1_std 16.11568495246132 type value nauc_ndcg_at_20_diff1 20.55696079927241 type value nauc_ndcg_at_20_max 37.60669992563356 type value nauc_ndcg_at_20_std 21.09713313195671 type value nauc_ndcg_at_3_diff1 22.08438430773322 type value nauc_ndcg_at_3_max 32.68110059834722 type value nauc_ndcg_at_3_std 15.267429669015595 type value nauc_ndcg_at_5_diff1 21.715020935808575 type value nauc_ndcg_at_5_max 35.17110301407326 type value nauc_ndcg_at_5_std 16.78243466311895 type value nauc_precision_at_1000_diff1 -3.2231794613702007 type value nauc_precision_at_1000_max 10.42559310530991 type value nauc_precision_at_1000_std 24.602086786850514 type value nauc_precision_at_100_diff1 1.7021223120345566 type value nauc_precision_at_100_max 17.38852629914526 type value nauc_precision_at_100_std 29.337128327095286 type value nauc_precision_at_10_diff1 12.164922485567033 type value nauc_precision_at_10_max 32.37319082664107 type value nauc_precision_at_10_std 26.300541100072984 type value nauc_precision_at_1_diff1 26.355328110953575 type value nauc_precision_at_1_max 30.471508547730092 type value nauc_precision_at_1_std 16.11568495246132 type value nauc_precision_at_20_diff1 7.385735474290768 type value nauc_precision_at_20_max 28.422173054750115 type value nauc_precision_at_20_std 27.035109636511876 type value nauc_precision_at_3_diff1 16.418314508072836 type value nauc_precision_at_3_max 31.785139366157615 type value nauc_precision_at_3_std 20.32896371836789 type value nauc_precision_at_5_diff1 14.937559885788062 type value nauc_precision_at_5_max 32.24391988837453 type value nauc_precision_at_5_std 23.17707476156323 type value nauc_recall_at_1000_diff1 5.616430433184691 type value nauc_recall_at_1000_max 36.55384286718441 type value nauc_recall_at_1000_std 38.50298604014725 type value nauc_recall_at_100_diff1 8.877636292128273 type value nauc_recall_at_100_max 30.860213540250705 type value nauc_recall_at_100_std 28.929321541751467 type value nauc_recall_at_10_diff1 16.07834176997954 type value nauc_recall_at_10_max 35.937627989165364 type value nauc_recall_at_10_std 18.808606461025498 type value nauc_recall_at_1_diff1 28.652249616122717 type value nauc_recall_at_1_max 32.05131359795648 type value nauc_recall_at_1_std 11.262948253532807 type value nauc_recall_at_20_diff1 12.600911526162099 type value nauc_recall_at_20_max 35.177943309574985 type value nauc_recall_at_20_std 21.99092004265232 type value nauc_recall_at_3_diff1 17.49507952659312 type value nauc_recall_at_3_max 31.406559780417105 type value nauc_recall_at_3_std 12.274503076493051 type value nauc_recall_at_5_diff1 16.612956574037305 type value nauc_recall_at_5_max 33.34670088062603 type value nauc_recall_at_5_std 14.445553526736607 type value ndcg_at_1 24.365000000000002 type value ndcg_at_10 27.794999999999998 type value ndcg_at_100 35.11 type value ndcg_at_1000 38.383 type value ndcg_at_20 30.616 type value ndcg_at_3 21.97 type value ndcg_at_5 24.264 type value precision_at_1 24.365000000000002 type value precision_at_10 8.827 type value precision_at_100 1.6660000000000001 type value precision_at_1000 0.22799999999999998 type value precision_at_20 5.6160000000000005 type value precision_at_3 16.2 type value precision_at_5 13.055 type value recall_at_1 11.138 type value recall_at_10 34.454 type value recall_at_100 59.648 type value recall_at_1000 77.823 type value recall_at_20 42.476 type value recall_at_3 20.630000000000003 type value recall_at_5 26.517000000000003 type Retrieval dataset metrics task config name revision split type default MTEB DBPedia c0f706b76e590d620bd6618b3ca8efdd34e2d659 test mteb/dbpedia type value main_score 39.582 type value map_at_1 8.193 type value map_at_10 18.838 type value map_at_100 26.791999999999998 type value map_at_1000 28.659000000000002 type value map_at_20 21.678 type value map_at_3 13.535 type value map_at_5 15.706000000000001 type value mrr_at_1 61.25000000000001 type value mrr_at_10 71.5827380952381 type value mrr_at_100 71.92227940484834 type value mrr_at_1000 71.92843656364919 type value mrr_at_20 71.82391254578756 type value mrr_at_3 69.54166666666667 type value mrr_at_5 70.89166666666667 type value nauc_map_at_1000_diff1 20.81525104511085 type value nauc_map_at_1000_max 12.28738487676873 type value nauc_map_at_1000_std 24.87551199629768 type value nauc_map_at_100_diff1 21.693837182713217 type value nauc_map_at_100_max 8.69725977707396 type value nauc_map_at_100_std 21.354633072475515 type value nauc_map_at_10_diff1 24.388731902741767 type value nauc_map_at_10_max -4.0866423282629585 type value nauc_map_at_10_std -0.9510081645949322 type value nauc_map_at_1_diff1 32.58191575261803 type value nauc_map_at_1_max -10.57813486927926 type value nauc_map_at_1_std -9.588423425329879 type value nauc_map_at_20_diff1 24.050743021827124 type value nauc_map_at_20_max 0.6686240161106345 type value nauc_map_at_20_std 6.53795559839344 type value nauc_map_at_3_diff1 26.43827919066607 type value nauc_map_at_3_max -10.727017270257825 type value nauc_map_at_3_std -9.512078389268677 type value nauc_map_at_5_diff1 25.71002404847907 type value nauc_map_at_5_max -7.097015507701878 type value nauc_map_at_5_std -6.476602516100202 type value nauc_mrr_at_1000_diff1 45.608034553728835 type value nauc_mrr_at_1000_max 30.922028122514266 type value nauc_mrr_at_1000_std 34.21750207725521 type value nauc_mrr_at_100_diff1 45.590642197534805 type value nauc_mrr_at_100_max 30.930031708368194 type value nauc_mrr_at_100_std 34.21945637610545 type value nauc_mrr_at_10_diff1 45.540994123130126 type value nauc_mrr_at_10_max 30.83734303048343 type value nauc_mrr_at_10_std 34.348404162478694 type value nauc_mrr_at_1_diff1 49.560483335546415 type value nauc_mrr_at_1_max 28.883661816871232 type value nauc_mrr_at_1_std 30.89553654418874 type value nauc_mrr_at_20_diff1 45.499322734057515 type value nauc_mrr_at_20_max 30.918972161205733 type value nauc_mrr_at_20_std 34.282904222510595 type value nauc_mrr_at_3_diff1 45.39622724954005 type value nauc_mrr_at_3_max 31.457074078677454 type value nauc_mrr_at_3_std 34.079043384571555 type value nauc_mrr_at_5_diff1 44.71358730464237 type value nauc_mrr_at_5_max 30.69295376764748 type value nauc_mrr_at_5_std 34.31128800389916 type value nauc_ndcg_at_1000_diff1 23.109017019057422 type value nauc_ndcg_at_1000_max 23.08462483716398 type value nauc_ndcg_at_1000_std 36.8911815972109 type value nauc_ndcg_at_100_diff1 23.827280037818173 type value nauc_ndcg_at_100_max 13.309666633249211 type value nauc_ndcg_at_100_std 28.44384667395871 type value nauc_ndcg_at_10_diff1 26.972856731999386 type value nauc_ndcg_at_10_max 14.620707258357266 type value nauc_ndcg_at_10_std 23.111341368346462 type value nauc_ndcg_at_1_diff1 43.59088178770794 type value nauc_ndcg_at_1_max 21.904917923054317 type value nauc_ndcg_at_1_std 21.98647522718905 type value nauc_ndcg_at_20_diff1 26.283361626051914 type value nauc_ndcg_at_20_max 11.10518046266052 type value nauc_ndcg_at_20_std 21.355473505613944 type value nauc_ndcg_at_3_diff1 30.024148446672083 type value nauc_ndcg_at_3_max 18.48737788479935 type value nauc_ndcg_at_3_std 23.24967559220411 type value nauc_ndcg_at_5_diff1 27.31687195788342 type value nauc_ndcg_at_5_max 17.233426051712428 type value nauc_ndcg_at_5_std 22.98467702068255 type value nauc_precision_at_1000_diff1 -13.448141290306074 type value nauc_precision_at_1000_max 42.26049965587544 type value nauc_precision_at_1000_std 17.838997647650835 type value nauc_precision_at_100_diff1 -5.070670934466766 type value nauc_precision_at_100_max 33.96276536553548 type value nauc_precision_at_100_std 47.592571562595765 type value nauc_precision_at_10_diff1 5.079452111840327 type value nauc_precision_at_10_max 33.145301874068146 type value nauc_precision_at_10_std 46.26256386765269 type value nauc_precision_at_1_diff1 49.560483335546415 type value nauc_precision_at_1_max 28.883661816871232 type value nauc_precision_at_1_std 30.89553654418874 type value nauc_precision_at_20_diff1 3.253674888888517 type value nauc_precision_at_20_max 34.667104498369575 type value nauc_precision_at_20_std 49.202859485875535 type value nauc_precision_at_3_diff1 15.790066053828234 type value nauc_precision_at_3_max 27.215083484496542 type value nauc_precision_at_3_std 33.11505410450215 type value nauc_precision_at_5_diff1 9.530674873702113 type value nauc_precision_at_5_max 31.21998248355014 type value nauc_precision_at_5_std 39.07247161423012 type value nauc_recall_at_1000_diff1 5.70231960458697 type value nauc_recall_at_1000_max 16.173798281531525 type value nauc_recall_at_1000_std 40.45772368713694 type value nauc_recall_at_100_diff1 9.815485122352673 type value nauc_recall_at_100_max 3.5894004884530735 type value nauc_recall_at_100_std 23.442799836302864 type value nauc_recall_at_10_diff1 14.537879655467389 type value nauc_recall_at_10_max -10.56087357341994 type value nauc_recall_at_10_std -7.372934296480146 type value nauc_recall_at_1_diff1 32.58191575261803 type value nauc_recall_at_1_max -10.57813486927926 type value nauc_recall_at_1_std -9.588423425329879 type value nauc_recall_at_20_diff1 13.359604621352824 type value nauc_recall_at_20_max -6.037674048018859 type value nauc_recall_at_20_std -0.191231970406073 type value nauc_recall_at_3_diff1 20.620776298724362 type value nauc_recall_at_3_max -14.34692846751201 type value nauc_recall_at_3_std -12.202460021792232 type value nauc_recall_at_5_diff1 17.573424943863706 type value nauc_recall_at_5_max -10.968843043485661 type value nauc_recall_at_5_std -10.513373048008399 type value ndcg_at_1 48.375 type value ndcg_at_10 39.582 type value ndcg_at_100 45.259 type value ndcg_at_1000 53.022000000000006 type value ndcg_at_20 39.038000000000004 type value ndcg_at_3 42.802 type value ndcg_at_5 40.538000000000004 type value precision_at_1 61.25000000000001 type value precision_at_10 32.2 type value precision_at_100 10.545 type value precision_at_1000 2.2880000000000003 type value precision_at_20 24.05 type value precision_at_3 48.083 type value precision_at_5 40.65 type value recall_at_1 8.193 type value recall_at_10 25.519 type value recall_at_100 54.124 type value recall_at_1000 77.92099999999999 type value recall_at_20 32.385999999999996 type value recall_at_3 15.211 type value recall_at_5 18.891 type Retrieval dataset metrics task config name revision split type default MTEB EmotionClassification 4f58c6b202a23cf9a4da393831edf4f9183cad37 test mteb/emotion type value accuracy 85.565 type value f1 81.12346731656551 type value f1_weighted 85.98372374550102 type value main_score 85.565 type Classification dataset metrics task config name revision split type default MTEB FEVER bea83ef9e8fb933d90a2f1d5515737465d613e12 test mteb/fever type value main_score 86.026 type value map_at_1 73.339 type value map_at_10 81.943 type value map_at_100 82.12899999999999 type value map_at_1000 82.145 type value map_at_20 82.05799999999999 type value map_at_3 80.827 type value map_at_5 81.628 type value mrr_at_1 78.98289828982898 type value mrr_at_10 87.04412703175062 type value mrr_at_100 87.1023996343652 type value mrr_at_1000 87.10370910386118 type value mrr_at_20 87.08615223713309 type value mrr_at_3 86.2386238623861 type value mrr_at_5 86.86568656865666 type value nauc_map_at_1000_diff1 48.22616948132843 type value nauc_map_at_1000_max 1.6340021380561394 type value nauc_map_at_1000_std -25.200746351372793 type value nauc_map_at_100_diff1 48.198187398812806 type value nauc_map_at_100_max 1.6220191408228601 type value nauc_map_at_100_std -25.193042721137566 type value nauc_map_at_10_diff1 48.00585391806132 type value nauc_map_at_10_max 1.4817376575907626 type value nauc_map_at_10_std -25.201484788329843 type value nauc_map_at_1_diff1 52.76212226788538 type value nauc_map_at_1_max 0.0520314144507071 type value nauc_map_at_1_std -26.20833932232049 type value nauc_map_at_20_diff1 48.12533777970878 type value nauc_map_at_20_max 1.5240294773565493 type value nauc_map_at_20_std -25.192450618181123 type value nauc_map_at_3_diff1 47.96480519565094 type value nauc_map_at_3_max 1.1887774816136902 type value nauc_map_at_3_std -26.31363833371711 type value nauc_map_at_5_diff1 47.79080333430883 type value nauc_map_at_5_max 1.6220551876503297 type value nauc_map_at_5_std -25.250585439913415 type value nauc_mrr_at_1000_diff1 64.95992140968579 type value nauc_mrr_at_1000_max 1.6737288643493216 type value nauc_mrr_at_1000_std -37.732249646223224 type value nauc_mrr_at_100_diff1 64.95845005240741 type value nauc_mrr_at_100_max 1.6807060884331666 type value nauc_mrr_at_100_std -37.73314881154047 type value nauc_mrr_at_10_diff1 64.9115307834577 type value nauc_mrr_at_10_max 1.7195209183889257 type value nauc_mrr_at_10_std -37.88536525017639 type value nauc_mrr_at_1_diff1 66.13713227430745 type value nauc_mrr_at_1_max 0.37082095312916874 type value nauc_mrr_at_1_std -34.379038222842254 type value nauc_mrr_at_20_diff1 64.95488651854674 type value nauc_mrr_at_20_max 1.6985375216432168 type value nauc_mrr_at_20_std -37.755703989608705 type value nauc_mrr_at_3_diff1 64.9535677343948 type value nauc_mrr_at_3_max 1.5195414353630512 type value nauc_mrr_at_3_std -39.21735562852805 type value nauc_mrr_at_5_diff1 64.85513437757459 type value nauc_mrr_at_5_max 1.9382830256224208 type value nauc_mrr_at_5_std -38.043842104083545 type value nauc_ndcg_at_1000_diff1 49.74095915307536 type value nauc_ndcg_at_1000_max 2.605169283095937 type value nauc_ndcg_at_1000_std -25.835814259340832 type value nauc_ndcg_at_100_diff1 49.002859024867945 type value nauc_ndcg_at_100_max 2.5116469969385884 type value nauc_ndcg_at_100_std -25.479921013562272 type value nauc_ndcg_at_10_diff1 48.25197176801494 type value nauc_ndcg_at_10_max 1.9108104946028264 type value nauc_ndcg_at_10_std -25.780784974391295 type value nauc_ndcg_at_1_diff1 66.13713227430745 type value nauc_ndcg_at_1_max 0.37082095312916874 type value nauc_ndcg_at_1_std -34.379038222842254 type value nauc_ndcg_at_20_diff1 48.59674729644139 type value nauc_ndcg_at_20_max 1.9950884849133927 type value nauc_ndcg_at_20_std -25.569135598052622 type value nauc_ndcg_at_3_diff1 49.305511135576275 type value nauc_ndcg_at_3_max 1.8638668857901368 type value nauc_ndcg_at_3_std -29.02269314595723 type value nauc_ndcg_at_5_diff1 48.1680764938404 type value nauc_ndcg_at_5_max 2.4842182285117964 type value nauc_ndcg_at_5_std -26.244542780767375 type value nauc_precision_at_1000_diff1 -4.478420343136971 type value nauc_precision_at_1000_max 11.70949232501659 type value nauc_precision_at_1000_std 1.7386198733671119 type value nauc_precision_at_100_diff1 -4.172269763651759 type value nauc_precision_at_100_max 13.082661117154743 type value nauc_precision_at_100_std 1.8002212793127355 type value nauc_precision_at_10_diff1 5.702289274109695 type value nauc_precision_at_10_max 8.484620250928458 type value nauc_precision_at_10_std -8.132389694515703 type value nauc_precision_at_1_diff1 66.13713227430745 type value nauc_precision_at_1_max 0.37082095312916874 type value nauc_precision_at_1_std -34.379038222842254 type value nauc_precision_at_20_diff1 0.5564831263316283 type value nauc_precision_at_20_max 8.881191911131173 type value nauc_precision_at_20_std -3.696180671957281 type value nauc_precision_at_3_diff1 35.75913314270679 type value nauc_precision_at_3_max 7.896253718358011 type value nauc_precision_at_3_std -33.8336411888768 type value nauc_precision_at_5_diff1 17.101795422527648 type value nauc_precision_at_5_max 11.993885038446976 type value nauc_precision_at_5_std -16.39044303210142 type value nauc_recall_at_1000_diff1 1.765610982286282 type value nauc_recall_at_1000_max 16.0490507693684 type value nauc_recall_at_1000_std 28.474043694387696 type value nauc_recall_at_100_diff1 6.2725603406909265 type value nauc_recall_at_100_max 10.665282199745704 type value nauc_recall_at_100_std 13.266482323582757 type value nauc_recall_at_10_diff1 16.010002473322103 type value nauc_recall_at_10_max 4.051158641772395 type value nauc_recall_at_10_std -3.963886778602456 type value nauc_recall_at_1_diff1 52.76212226788538 type value nauc_recall_at_1_max 0.0520314144507071 type value nauc_recall_at_1_std -26.20833932232049 type value nauc_recall_at_20_diff1 12.763325751516286 type value nauc_recall_at_20_max 4.589618045061225 type value nauc_recall_at_20_std 2.3135711002947525 type value nauc_recall_at_3_diff1 31.878202992328298 type value nauc_recall_at_3_max 2.398044119809843 type value nauc_recall_at_3_std -22.48228292127779 type value nauc_recall_at_5_diff1 22.01091185405021 type value nauc_recall_at_5_max 6.161863454884261 type value nauc_recall_at_5_std -10.442113305092082 type value ndcg_at_1 78.983 type value ndcg_at_10 86.026 type value ndcg_at_100 86.666 type value ndcg_at_1000 86.945 type value ndcg_at_20 86.333 type value ndcg_at_3 84.269 type value ndcg_at_5 85.439 type value precision_at_1 78.983 type value precision_at_10 10.282 type value precision_at_100 1.078 type value precision_at_1000 0.11199999999999999 type value precision_at_20 5.2330000000000005 type value precision_at_3 32.218 type value precision_at_5 20.06 type value recall_at_1 73.339 type value recall_at_10 93.557 type value recall_at_100 96.03399999999999 type value recall_at_1000 97.784 type value recall_at_20 94.6 type value recall_at_3 88.851 type value recall_at_5 91.81 type Retrieval dataset metrics task config name revision split type default MTEB FiQA2018 27a168819829fe9bcd655c2df245fb19452e8e06 test mteb/fiqa type value main_score 45.019 type value map_at_1 21.923000000000002 type value map_at_10 36.661 type value map_at_100 38.727000000000004 type value map_at_1000 38.896 type value map_at_20 37.821 type value map_at_3 31.812 type value map_at_5 34.474 type value mrr_at_1 43.05555555555556 type value mrr_at_10 52.714824612972755 type value mrr_at_100 53.47543808894285 type value mrr_at_1000 53.50616025822894 type value mrr_at_20 53.14543059863263 type value mrr_at_3 50.10288065843621 type value mrr_at_5 51.715534979423836 type value nauc_map_at_1000_diff1 41.776705312708486 type value nauc_map_at_1000_max 24.93532754336337 type value nauc_map_at_1000_std -2.794190590614799 type value nauc_map_at_100_diff1 41.73579109673881 type value nauc_map_at_100_max 24.80625280860252 type value nauc_map_at_100_std -2.814441295874619 type value nauc_map_at_10_diff1 41.75395538260581 type value nauc_map_at_10_max 23.219207680303324 type value nauc_map_at_10_std -3.5779070328036138 type value nauc_map_at_1_diff1 48.46545399169614 type value nauc_map_at_1_max 16.49315969594624 type value nauc_map_at_1_std -7.505787454483636 type value nauc_map_at_20_diff1 41.53641801097531 type value nauc_map_at_20_max 24.00770569213574 type value nauc_map_at_20_std -3.191754163523877 type value nauc_map_at_3_diff1 41.75052616046243 type value nauc_map_at_3_max 19.115081667001014 type value nauc_map_at_3_std -6.668596004487064 type value nauc_map_at_5_diff1 42.45446754604312 type value nauc_map_at_5_max 20.947253345126185 type value nauc_map_at_5_std -5.125992439200763 type value nauc_mrr_at_1000_diff1 52.09717084990717 type value nauc_mrr_at_1000_max 38.086957556354456 type value nauc_mrr_at_1000_std -0.68079244284855 type value nauc_mrr_at_100_diff1 52.081504543550686 type value nauc_mrr_at_100_max 38.10189737899758 type value nauc_mrr_at_100_std -0.6731400759499799 type value nauc_mrr_at_10_diff1 51.962775327926934 type value nauc_mrr_at_10_max 37.860734658269976 type value nauc_mrr_at_10_std -0.8627588620266099 type value nauc_mrr_at_1_diff1 56.643374967422865 type value nauc_mrr_at_1_max 37.424164231372195 type value nauc_mrr_at_1_std -3.808604224746232 type value nauc_mrr_at_20_diff1 51.9634718440668 type value nauc_mrr_at_20_max 37.99992134394818 type value nauc_mrr_at_20_std -0.5725435512805715 type value nauc_mrr_at_3_diff1 51.9083290591896 type value nauc_mrr_at_3_max 37.49495462369628 type value nauc_mrr_at_3_std -2.193915400523023 type value nauc_mrr_at_5_diff1 52.24074329239152 type value nauc_mrr_at_5_max 37.96365352861984 type value nauc_mrr_at_5_std -1.5116002789297864 type value nauc_ndcg_at_1000_diff1 43.88564426048843 type value nauc_ndcg_at_1000_max 31.371070838376326 type value nauc_ndcg_at_1000_std 1.182058822041445 type value nauc_ndcg_at_100_diff1 43.47882005622348 type value nauc_ndcg_at_100_max 30.23626893448966 type value nauc_ndcg_at_100_std 1.3554256181078206 type value nauc_ndcg_at_10_diff1 42.78328747987686 type value nauc_ndcg_at_10_max 26.971284497406334 type value nauc_ndcg_at_10_std -0.9361763271905158 type value nauc_ndcg_at_1_diff1 56.643374967422865 type value nauc_ndcg_at_1_max 37.424164231372195 type value nauc_ndcg_at_1_std -3.808604224746232 type value nauc_ndcg_at_20_diff1 42.51200178317055 type value nauc_ndcg_at_20_max 27.807479427212844 type value nauc_ndcg_at_20_std -0.16279719845344157 type value nauc_ndcg_at_3_diff1 41.983935082179556 type value nauc_ndcg_at_3_max 28.446235814415143 type value nauc_ndcg_at_3_std -3.0007943000595003 type value nauc_ndcg_at_5_diff1 43.21852196702825 type value nauc_ndcg_at_5_max 26.601248066336986 type value nauc_ndcg_at_5_std -2.5471886292781702 type value nauc_precision_at_1000_diff1 -0.26010199321259797 type value nauc_precision_at_1000_max 35.79601474558423 type value nauc_precision_at_1000_std 14.342818001909988 type value nauc_precision_at_100_diff1 6.004698224173632 type value nauc_precision_at_100_max 38.52857855255943 type value nauc_precision_at_100_std 16.21705591642149 type value nauc_precision_at_10_diff1 17.49728453546782 type value nauc_precision_at_10_max 38.24671033647839 type value nauc_precision_at_10_std 12.030940471652098 type value nauc_precision_at_1_diff1 56.643374967422865 type value nauc_precision_at_1_max 37.424164231372195 type value nauc_precision_at_1_std -3.808604224746232 type value nauc_precision_at_20_diff1 13.057739432783794 type value nauc_precision_at_20_max 37.84177604877064 type value nauc_precision_at_20_std 13.135243737603359 type value nauc_precision_at_3_diff1 29.106393446078787 type value nauc_precision_at_3_max 33.51402929333319 type value nauc_precision_at_3_std 1.9298573035534488 type value nauc_precision_at_5_diff1 25.039378213923403 type value nauc_precision_at_5_max 36.213261098065125 type value nauc_precision_at_5_std 7.142334933169122 type value nauc_recall_at_1000_diff1 24.897608581023757 type value nauc_recall_at_1000_max 24.60932291382376 type value nauc_recall_at_1000_std 30.05990115014322 type value nauc_recall_at_100_diff1 30.807527684131564 type value nauc_recall_at_100_max 22.540558835740985 type value nauc_recall_at_100_std 14.493739358980907 type value nauc_recall_at_10_diff1 31.683742260409076 type value nauc_recall_at_10_max 17.828711448272134 type value nauc_recall_at_10_std 1.899605838015785 type value nauc_recall_at_1_diff1 48.46545399169614 type value nauc_recall_at_1_max 16.49315969594624 type value nauc_recall_at_1_std -7.505787454483636 type value nauc_recall_at_20_diff1 30.08305577595204 type value nauc_recall_at_20_max 18.75062281011906 type value nauc_recall_at_20_std 4.502661433146342 type value nauc_recall_at_3_diff1 33.53153516576839 type value nauc_recall_at_3_max 14.790607412204485 type value nauc_recall_at_3_std -6.1140323409194846 type value nauc_recall_at_5_diff1 35.64279484984148 type value nauc_recall_at_5_max 15.401875599379574 type value nauc_recall_at_5_std -3.2844856697915774 type value ndcg_at_1 43.056 type value ndcg_at_10 45.019 type value ndcg_at_100 51.98199999999999 type value ndcg_at_1000 54.581999999999994 type value ndcg_at_20 47.721999999999994 type value ndcg_at_3 40.54 type value ndcg_at_5 42.142 type value precision_at_1 43.056 type value precision_at_10 12.531 type value precision_at_100 1.9949999999999999 type value precision_at_1000 0.245 type value precision_at_20 7.446 type value precision_at_3 27.058 type value precision_at_5 20.061999999999998 type value recall_at_1 21.923000000000002 type value recall_at_10 52.85300000000001 type value recall_at_100 78.133 type value recall_at_1000 93.75 type value recall_at_20 61.085 type value recall_at_3 37.118 type value recall_at_5 44.031 type Retrieval dataset metrics task config name revision split type default MTEB HotpotQA ab518f4d6fcca38d87c25209f94beba119d02014 test mteb/hotpotqa type value main_score 68.65299999999999 type value map_at_1 38.893 type value map_at_10 59.375 type value map_at_100 60.303 type value map_at_1000 60.364 type value map_at_20 59.964 type value map_at_3 55.718 type value map_at_5 57.99999999999999 type value mrr_at_1 77.78528021607022 type value mrr_at_10 84.49470006323453 type value mrr_at_100 84.6519637218647 type value mrr_at_1000 84.65768034160618 type value mrr_at_20 84.61055874712832 type value mrr_at_3 83.59441818591013 type value mrr_at_5 84.19266261534956 type value nauc_map_at_1000_diff1 15.948378928650673 type value nauc_map_at_1000_max 15.711635353869994 type value nauc_map_at_1000_std 0.937019577383957 type value nauc_map_at_100_diff1 15.918426215773247 type value nauc_map_at_100_max 15.699627284031124 type value nauc_map_at_100_std 0.9584857374941618 type value nauc_map_at_10_diff1 15.879270822613408 type value nauc_map_at_10_max 15.463063162099125 type value nauc_map_at_10_std 0.15481877422177437 type value nauc_map_at_1_diff1 71.30652188008001 type value nauc_map_at_1_max 32.60802008342313 type value nauc_map_at_1_std -12.29496015891874 type value nauc_map_at_20_diff1 15.853758892635852 type value nauc_map_at_20_max 15.570900027569573 type value nauc_map_at_20_std 0.6783433634347852 type value nauc_map_at_3_diff1 17.97014394473015 type value nauc_map_at_3_max 15.218485551181926 type value nauc_map_at_3_std -2.4303445320319272 type value nauc_map_at_5_diff1 16.50404017618271 type value nauc_map_at_5_max 15.285663669100073 type value nauc_map_at_5_std -0.989351556289713 type value nauc_mrr_at_1000_diff1 70.0763435325149 type value nauc_mrr_at_1000_max 34.01106818267054 type value nauc_mrr_at_1000_std -10.558570244805534 type value nauc_mrr_at_100_diff1 70.0763826742575 type value nauc_mrr_at_100_max 34.01329127860268 type value nauc_mrr_at_100_std -10.553859035770314 type value nauc_mrr_at_10_diff1 70.03690200308235 type value nauc_mrr_at_10_max 34.10786779680883 type value nauc_mrr_at_10_std -10.509981664609755 type value nauc_mrr_at_1_diff1 71.30652188008001 type value nauc_mrr_at_1_max 32.60802008342313 type value nauc_mrr_at_1_std -12.29496015891874 type value nauc_mrr_at_20_diff1 70.07320564989382 type value nauc_mrr_at_20_max 34.01911070550699 type value nauc_mrr_at_20_std -10.532501476325248 type value nauc_mrr_at_3_diff1 69.73518331018965 type value nauc_mrr_at_3_max 33.7438084424745 type value nauc_mrr_at_3_std -11.302692900313119 type value nauc_mrr_at_5_diff1 69.86565354847778 type value nauc_mrr_at_5_max 34.135593857390504 type value nauc_mrr_at_5_std -10.380178093077621 type value nauc_ndcg_at_1000_diff1 20.865436555566845 type value nauc_ndcg_at_1000_max 18.83121871269731 type value nauc_ndcg_at_1000_std 3.566623532300052 type value nauc_ndcg_at_100_diff1 19.90357263881322 type value nauc_ndcg_at_100_max 18.387111355628193 type value nauc_ndcg_at_100_std 4.243680531655493 type value nauc_ndcg_at_10_diff1 19.721051339510907 type value nauc_ndcg_at_10_max 17.558512453515227 type value nauc_ndcg_at_10_std 1.2891095080720567 type value nauc_ndcg_at_1_diff1 71.30652188008001 type value nauc_ndcg_at_1_max 32.60802008342313 type value nauc_ndcg_at_1_std -12.29496015891874 type value nauc_ndcg_at_20_diff1 19.519425870891023 type value nauc_ndcg_at_20_max 17.77152674804043 type value nauc_ndcg_at_20_std 2.7253915106561712 type value nauc_ndcg_at_3_diff1 23.595619290089495 type value nauc_ndcg_at_3_max 17.443501928111456 type value nauc_ndcg_at_3_std -3.1185231896019183 type value nauc_ndcg_at_5_diff1 21.128676475251222 type value nauc_ndcg_at_5_max 17.427440887891148 type value nauc_ndcg_at_5_std -0.8006655617871765 type value nauc_precision_at_1000_diff1 -18.605360521020412 type value nauc_precision_at_1000_max 13.992651128348118 type value nauc_precision_at_1000_std 34.896942379633316 type value nauc_precision_at_100_diff1 -11.425102107370272 type value nauc_precision_at_100_max 11.216164840931667 type value nauc_precision_at_100_std 27.722125456439343 type value nauc_precision_at_10_diff1 -3.1401539776631653 type value nauc_precision_at_10_max 10.416214004945402 type value nauc_precision_at_10_std 10.251563605515335 type value nauc_precision_at_1_diff1 71.30652188008001 type value nauc_precision_at_1_max 32.60802008342313 type value nauc_precision_at_1_std -12.29496015891874 type value nauc_precision_at_20_diff1 -6.456921653790667 type value nauc_precision_at_20_max 10.23022445081364 type value nauc_precision_at_20_std 15.935771905722302 type value nauc_precision_at_3_diff1 8.38156786039047 type value nauc_precision_at_3_max 12.08129239567508 type value nauc_precision_at_3_std 0.05626041327325479 type value nauc_precision_at_5_diff1 2.4102262974666653 type value nauc_precision_at_5_max 11.160384909564122 type value nauc_precision_at_5_std 4.587163311214582 type value nauc_recall_at_1000_diff1 -18.605360521019925 type value nauc_recall_at_1000_max 13.992651128348363 type value nauc_recall_at_1000_std 34.89694237963353 type value nauc_recall_at_100_diff1 -11.425102107370193 type value nauc_recall_at_100_max 11.216164840931476 type value nauc_recall_at_100_std 27.72212545643919 type value nauc_recall_at_10_diff1 -3.140153977663016 type value nauc_recall_at_10_max 10.416214004945413 type value nauc_recall_at_10_std 10.251563605515395 type value nauc_recall_at_1_diff1 71.30652188008001 type value nauc_recall_at_1_max 32.60802008342313 type value nauc_recall_at_1_std -12.29496015891874 type value nauc_recall_at_20_diff1 -6.45692165379055 type value nauc_recall_at_20_max 10.230224450813735 type value nauc_recall_at_20_std 15.935771905722335 type value nauc_recall_at_3_diff1 8.381567860390362 type value nauc_recall_at_3_max 12.081292395675078 type value nauc_recall_at_3_std 0.05626041327321052 type value nauc_recall_at_5_diff1 2.4102262974666355 type value nauc_recall_at_5_max 11.160384909564078 type value nauc_recall_at_5_std 4.587163311214529 type value ndcg_at_1 77.78500000000001 type value ndcg_at_10 68.65299999999999 type value ndcg_at_100 71.69200000000001 type value ndcg_at_1000 72.869 type value ndcg_at_20 70.078 type value ndcg_at_3 63.568000000000005 type value ndcg_at_5 66.402 type value precision_at_1 77.78500000000001 type value precision_at_10 14.386 type value precision_at_100 1.672 type value precision_at_1000 0.183 type value precision_at_20 7.6499999999999995 type value precision_at_3 40.473 type value precision_at_5 26.515 type value recall_at_1 38.893 type value recall_at_10 71.931 type value recall_at_100 83.619 type value recall_at_1000 91.431 type value recall_at_20 76.496 type value recall_at_3 60.709 type value recall_at_5 66.286 type Retrieval dataset metrics task config name revision split type default MTEB ImdbClassification 3d86128a09e091d6018b6d26cad27f2739fc2db7 test mteb/imdb type value accuracy 95.0268 type value ap 92.72653250341486 type value ap_weighted 92.72653250341486 type value f1 95.02503365717179 type value f1_weighted 95.02503365717179 type value main_score 95.0268 type Classification dataset metrics task config name revision split type default MTEB MSMARCO c5a29a104738b98a9e76336939199e264163d4a0 dev mteb/msmarco type value main_score 35.191 type value map_at_1 16.139 type value map_at_10 28.101 type value map_at_100 29.461 type value map_at_1000 29.515 type value map_at_20 28.936 type value map_at_3 23.954 type value map_at_5 26.308999999999997 type value mrr_at_1 16.59025787965616 type value mrr_at_10 28.583100241051344 type value mrr_at_100 29.89488944200741 type value mrr_at_1000 29.94198818201922 type value mrr_at_20 29.397153289126486 type value mrr_at_3 24.512893982807853 type value mrr_at_5 26.840974212034318 type value nauc_map_at_1000_diff1 29.92308133337915 type value nauc_map_at_1000_max -4.792013160789208 type value nauc_map_at_1000_std -20.365722765519205 type value nauc_map_at_100_diff1 29.927608009586475 type value nauc_map_at_100_max -4.813011061550381 type value nauc_map_at_100_std -20.34066079647475 type value nauc_map_at_10_diff1 29.85964417677257 type value nauc_map_at_10_max -5.020819297438392 type value nauc_map_at_10_std -21.185600868900707 type value nauc_map_at_1_diff1 31.91727354325134 type value nauc_map_at_1_max -3.3836191178002637 type value nauc_map_at_1_std -18.94420033626203 type value nauc_map_at_20_diff1 29.909409775064265 type value nauc_map_at_20_max -4.882624170262229 type value nauc_map_at_20_std -20.737422787243176 type value nauc_map_at_3_diff1 29.96619551770926 type value nauc_map_at_3_max -4.521984358305567 type value nauc_map_at_3_std -20.675567430573214 type value nauc_map_at_5_diff1 29.672157845793336 type value nauc_map_at_5_max -4.784226867946108 type value nauc_map_at_5_std -21.090554010504313 type value nauc_mrr_at_1000_diff1 29.57786251899136 type value nauc_mrr_at_1000_max -4.554864207268301 type value nauc_mrr_at_1000_std -20.124071230468733 type value nauc_mrr_at_100_diff1 29.57869911178864 type value nauc_mrr_at_100_max -4.568738533954914 type value nauc_mrr_at_100_std -20.097461372571754 type value nauc_mrr_at_10_diff1 29.50101055760309 type value nauc_mrr_at_10_max -4.699465165716407 type value nauc_mrr_at_10_std -20.85880213075095 type value nauc_mrr_at_1_diff1 31.5283761916309 type value nauc_mrr_at_1_max -3.2410968598060226 type value nauc_mrr_at_1_std -18.877804738741848 type value nauc_mrr_at_20_diff1 29.55469091898283 type value nauc_mrr_at_20_max -4.6114669798589585 type value nauc_mrr_at_20_std -20.433076769992457 type value nauc_mrr_at_3_diff1 29.62441465248462 type value nauc_mrr_at_3_max -4.317634456438896 type value nauc_mrr_at_3_std -20.545356421989975 type value nauc_mrr_at_5_diff1 29.3174731757817 type value nauc_mrr_at_5_max -4.524554398532275 type value nauc_mrr_at_5_std -20.87564955466439 type value nauc_ndcg_at_1000_diff1 29.417049449756306 type value nauc_ndcg_at_1000_max -4.429863573283831 type value nauc_ndcg_at_1000_std -18.672687178180762 type value nauc_ndcg_at_100_diff1 29.52545788575206 type value nauc_ndcg_at_100_max -4.839548635918072 type value nauc_ndcg_at_100_std -17.445902376477168 type value nauc_ndcg_at_10_diff1 29.349337034114708 type value nauc_ndcg_at_10_max -5.654575625474153 type value nauc_ndcg_at_10_std -21.867391862075433 type value nauc_ndcg_at_1_diff1 31.5283761916309 type value nauc_ndcg_at_1_max -3.2410968598060226 type value nauc_ndcg_at_1_std -18.877804738741848 type value nauc_ndcg_at_20_diff1 29.478679665234736 type value nauc_ndcg_at_20_max -5.348280869926551 type value nauc_ndcg_at_20_std -20.32251566103604 type value nauc_ndcg_at_3_diff1 29.41586840338385 type value nauc_ndcg_at_3_max -4.737448759293484 type value nauc_ndcg_at_3_std -21.114595209094198 type value nauc_ndcg_at_5_diff1 28.95897834819025 type value nauc_ndcg_at_5_max -5.144033504465505 type value nauc_ndcg_at_5_std -21.73482008242439 type value nauc_precision_at_1000_diff1 -4.773246418887565 type value nauc_precision_at_1000_max 18.94086713593158 type value nauc_precision_at_1000_std 14.940921913943725 type value nauc_precision_at_100_diff1 15.529104524208284 type value nauc_precision_at_100_max 4.152043132226839 type value nauc_precision_at_100_std 15.362588630598356 type value nauc_precision_at_10_diff1 26.327252473718293 type value nauc_precision_at_10_max -6.385696358427295 type value nauc_precision_at_10_std -22.43695468265468 type value nauc_precision_at_1_diff1 31.5283761916309 type value nauc_precision_at_1_max -3.2410968598060226 type value nauc_precision_at_1_std -18.877804738741848 type value nauc_precision_at_20_diff1 25.09386904802987 type value nauc_precision_at_20_max -4.384006847324815 type value nauc_precision_at_20_std -15.476174306633775 type value nauc_precision_at_3_diff1 27.88147581285313 type value nauc_precision_at_3_max -5.10330889992625 type value nauc_precision_at_3_std -22.17804890064486 type value nauc_precision_at_5_diff1 26.673260429548385 type value nauc_precision_at_5_max -5.849985467654149 type value nauc_precision_at_5_std -23.22704929951935 type value nauc_recall_at_1000_diff1 11.078337058729081 type value nauc_recall_at_1000_max 29.31329518339392 type value nauc_recall_at_1000_std 61.689932707089845 type value nauc_recall_at_100_diff1 27.694660226790095 type value nauc_recall_at_100_max -4.662880554456902 type value nauc_recall_at_100_std 17.291575712920476 type value nauc_recall_at_10_diff1 28.14620642731046 type value nauc_recall_at_10_max -7.883918071832969 type value nauc_recall_at_10_std -23.85382911185965 type value nauc_recall_at_1_diff1 31.91727354325134 type value nauc_recall_at_1_max -3.3836191178002637 type value nauc_recall_at_1_std -18.94420033626203 type value nauc_recall_at_20_diff1 28.411188230736368 type value nauc_recall_at_20_max -7.489052404904147 type value nauc_recall_at_20_std -17.923010929300084 type value nauc_recall_at_3_diff1 28.13888531840714 type value nauc_recall_at_3_max -5.385513963117635 type value nauc_recall_at_3_std -22.09635477229696 type value nauc_recall_at_5_diff1 27.197531472369057 type value nauc_recall_at_5_max -6.204044942502606 type value nauc_recall_at_5_std -23.25902678179945 type value ndcg_at_1 16.59 type value ndcg_at_10 35.191 type value ndcg_at_100 41.778999999999996 type value ndcg_at_1000 43.126999999999995 type value ndcg_at_20 38.153 type value ndcg_at_3 26.718999999999998 type value ndcg_at_5 30.919999999999998 type value precision_at_1 16.59 type value precision_at_10 5.992999999999999 type value precision_at_100 0.927 type value precision_at_1000 0.104 type value precision_at_20 3.6020000000000003 type value precision_at_3 11.815000000000001 type value precision_at_5 9.218 type value recall_at_1 16.139 type value recall_at_10 57.272999999999996 type value recall_at_100 87.819 type value recall_at_1000 98.10900000000001 type value recall_at_20 68.77 type value recall_at_3 34.172999999999995 type value recall_at_5 44.259 type Retrieval dataset metrics task config name revision split type en MTEB MTOPDomainClassification (en) d80d48c1eb48d3562165c59d59d0034df9fff0bf test mteb/mtop_domain type value accuracy 97.57637938896491 type value f1 97.39941554989736 type value f1_weighted 97.58495129362304 type value main_score 97.57637938896491 type Classification dataset metrics task config name revision split type en MTEB MTOPIntentClassification (en) ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba test mteb/mtop_intent type value accuracy 86.08071135430917 type value f1 60.67695519910473 type value f1_weighted 86.22253292076088 type value main_score 86.08071135430917 type Classification dataset metrics task config name revision split type en MTEB MassiveIntentClassification (en) 4672e20407010da34463acc759c162ca9734bca6 test mteb/amazon_massive_intent type value accuracy 74.9394754539341 type value f1 71.84595519829237 type value f1_weighted 73.7724380212837 type value main_score 74.9394754539341 type Classification dataset metrics task config name revision split type en MTEB MassiveScenarioClassification (en) fad2c6e8459f9e1c45d9315f4953d921437d70f8 test mteb/amazon_massive_scenario type value accuracy 82.0611970410222 type value f1 80.96764019308867 type value f1_weighted 81.75048816703206 type value main_score 82.0611970410222 type Classification dataset metrics task config name revision split type default MTEB MedrxivClusteringP2P e7a26af6f3ae46b30dde8737f02c07b1505bcc73 test mteb/medrxiv-clustering-p2p type value main_score 35.535315182381275 type value v_measure 35.535315182381275 type value v_measure_std 1.2947784991789062 type Clustering dataset metrics task config name revision split type default MTEB MedrxivClusteringS2S 35191c8c0dca72d8ff3efcd72aa802307d469663 test mteb/medrxiv-clustering-s2s type value main_score 32.701317380058356 type value v_measure 32.701317380058356 type value v_measure_std 1.212859415243672 type Clustering dataset metrics task config name revision split type default MTEB MindSmallReranking 59042f120c80e8afa9cdbb224f67076cec0fc9a7 test mteb/mind_small type value main_score 31.586146859630325 type value map 31.586146859630325 type value mrr 32.74920599119196 type value nAUC_map_diff1 11.669586995601716 type value nAUC_map_max -19.043343922416184 type value nAUC_map_std -0.002926267520007513 type value nAUC_mrr_diff1 11.132898797866952 type value nAUC_mrr_max -13.521554137760747 type value nAUC_mrr_std 1.6662256096686372 type Reranking dataset metrics task config name revision split type default MTEB NFCorpus ec0fa4fe99da2ff19ca1214b7966684033a58814 test mteb/nfcorpus type value main_score 34.493 type value map_at_1 5.469 type value map_at_10 12.681999999999999 type value map_at_100 16.136 type value map_at_1000 17.574 type value map_at_20 14.063 type value map_at_3 9.252 type value map_at_5 11.03 type value mrr_at_1 43.962848297213625 type value mrr_at_10 53.748095729519854 type value mrr_at_100 54.31371383821993 type value mrr_at_1000 54.34550446424 type value mrr_at_20 54.05753630252571 type value mrr_at_3 51.34158926728587 type value mrr_at_5 52.951496388028886 type value nauc_map_at_1000_diff1 22.42945451651053 type value nauc_map_at_1000_max 25.044939905094555 type value nauc_map_at_1000_std 14.6947376252321 type value nauc_map_at_100_diff1 24.05126858377848 type value nauc_map_at_100_max 24.260286968462943 type value nauc_map_at_100_std 11.274560706750162 type value nauc_map_at_10_diff1 28.610449405636412 type value nauc_map_at_10_max 17.669350840567517 type value nauc_map_at_10_std -0.5603965547026133 type value nauc_map_at_1_diff1 44.546139576048574 type value nauc_map_at_1_max 3.5966098414779686 type value nauc_map_at_1_std -15.204463497276185 type value nauc_map_at_20_diff1 26.93971089998854 type value nauc_map_at_20_max 20.89952744553902 type value nauc_map_at_20_std 4.323667205452283 type value nauc_map_at_3_diff1 34.03753780494977 type value nauc_map_at_3_max 10.951970261908517 type value nauc_map_at_3_std -8.942935860299977 type value nauc_map_at_5_diff1 31.13647526539977 type value nauc_map_at_5_max 13.55486409562657 type value nauc_map_at_5_std -6.285335121924455 type value nauc_mrr_at_1000_diff1 33.04380727929978 type value nauc_mrr_at_1000_max 40.97460730083534 type value nauc_mrr_at_1000_std 22.68307762886138 type value nauc_mrr_at_100_diff1 33.038505852668905 type value nauc_mrr_at_100_max 41.004813808229976 type value nauc_mrr_at_100_std 22.727078227914703 type value nauc_mrr_at_10_diff1 32.945102642427294 type value nauc_mrr_at_10_max 40.59087425732438 type value nauc_mrr_at_10_std 22.2969763977488 type value nauc_mrr_at_1_diff1 34.55355095202985 type value nauc_mrr_at_1_max 34.35691144716251 type value nauc_mrr_at_1_std 16.025738199559136 type value nauc_mrr_at_20_diff1 33.01684360381644 type value nauc_mrr_at_20_max 40.82433798731643 type value nauc_mrr_at_20_std 22.56838707992269 type value nauc_mrr_at_3_diff1 33.2000664328818 type value nauc_mrr_at_3_max 40.65557927809233 type value nauc_mrr_at_3_std 21.640445622194292 type value nauc_mrr_at_5_diff1 33.14724263980201 type value nauc_mrr_at_5_max 40.37502720649393 type value nauc_mrr_at_5_std 20.91483571628846 type value nauc_ndcg_at_1000_diff1 23.13999445390973 type value nauc_ndcg_at_1000_max 40.904356797688244 type value nauc_ndcg_at_1000_std 31.135131225973755 type value nauc_ndcg_at_100_diff1 21.60764588276507 type value nauc_ndcg_at_100_max 34.72455917031235 type value nauc_ndcg_at_100_std 26.084570343364895 type value nauc_ndcg_at_10_diff1 21.273666650824712 type value nauc_ndcg_at_10_max 36.42637032684147 type value nauc_ndcg_at_10_std 25.854371107614753 type value nauc_ndcg_at_1_diff1 35.40190534464431 type value nauc_ndcg_at_1_max 34.09394953710087 type value nauc_ndcg_at_1_std 15.082336268368568 type value nauc_ndcg_at_20_diff1 20.629683502494935 type value nauc_ndcg_at_20_max 35.01440571472175 type value nauc_ndcg_at_20_std 26.1516323412204 type value nauc_ndcg_at_3_diff1 27.314585132007803 type value nauc_ndcg_at_3_max 38.19301088947643 type value nauc_ndcg_at_3_std 22.37292581921333 type value nauc_ndcg_at_5_diff1 24.033794102904647 type value nauc_ndcg_at_5_max 36.466778291326506 type value nauc_ndcg_at_5_std 23.15763774408816 type value nauc_precision_at_1000_diff1 -13.984096369493178 type value nauc_precision_at_1000_max 8.50221544384146 type value nauc_precision_at_1000_std 35.62592696752026 type value nauc_precision_at_100_diff1 -12.115042643624523 type value nauc_precision_at_100_max 21.139964351279062 type value nauc_precision_at_100_std 45.41323150126541 type value nauc_precision_at_10_diff1 3.5604358960435594 type value nauc_precision_at_10_max 38.21371536948471 type value nauc_precision_at_10_std 40.093467246870674 type value nauc_precision_at_1_diff1 34.55355095202985 type value nauc_precision_at_1_max 34.35691144716251 type value nauc_precision_at_1_std 16.025738199559136 type value nauc_precision_at_20_diff1 -2.2994929672216142 type value nauc_precision_at_20_max 33.41182551515417 type value nauc_precision_at_20_std 42.926074063475376 type value nauc_precision_at_3_diff1 17.026846985190286 type value nauc_precision_at_3_max 40.78926087324481 type value nauc_precision_at_3_std 28.26154405706766 type value nauc_precision_at_5_diff1 10.066105504177528 type value nauc_precision_at_5_max 38.397299240351515 type value nauc_precision_at_5_std 31.504726528569105 type value nauc_recall_at_1000_diff1 5.433767085525343 type value nauc_recall_at_1000_max 17.082294989371675 type value nauc_recall_at_1000_std 17.867147762696924 type value nauc_recall_at_100_diff1 10.513494371628159 type value nauc_recall_at_100_max 19.63867418942476 type value nauc_recall_at_100_std 14.421450754520809 type value nauc_recall_at_10_diff1 22.750728383486376 type value nauc_recall_at_10_max 15.735611146890621 type value nauc_recall_at_10_std -0.40290229377136233 type value nauc_recall_at_1_diff1 44.546139576048574 type value nauc_recall_at_1_max 3.5966098414779686 type value nauc_recall_at_1_std -15.204463497276185 type value nauc_recall_at_20_diff1 22.44097500377964 type value nauc_recall_at_20_max 19.99783526750806 type value nauc_recall_at_20_std 5.831968175648315 type value nauc_recall_at_3_diff1 30.742501145388644 type value nauc_recall_at_3_max 11.887713348765457 type value nauc_recall_at_3_std -7.507756416467706 type value nauc_recall_at_5_diff1 25.251057623903268 type value nauc_recall_at_5_max 11.530971742020508 type value nauc_recall_at_5_std -6.9727238554804005 type value ndcg_at_1 42.57 type value ndcg_at_10 34.493 type value ndcg_at_100 31.912000000000003 type value ndcg_at_1000 40.485 type value ndcg_at_20 32.314 type value ndcg_at_3 39.546 type value ndcg_at_5 38.009 type value precision_at_1 43.963 type value precision_at_10 25.728 type value precision_at_100 8.297 type value precision_at_1000 2.094 type value precision_at_20 19.288 type value precision_at_3 37.564 type value precision_at_5 33.375 type value recall_at_1 5.469 type value recall_at_10 16.733 type value recall_at_100 32.867000000000004 type value recall_at_1000 63.873000000000005 type value recall_at_20 20.312 type value recall_at_3 10.386 type value recall_at_5 13.679 type Retrieval dataset metrics task config name revision split type default MTEB NQ b774495ed302d8c44a3a7ea25c90dbce03968f31 test mteb/nq type value main_score 49.539 type value map_at_1 26.016000000000002 type value map_at_10 41.23 type value map_at_100 42.466 type value map_at_1000 42.494 type value map_at_20 42.049 type value map_at_3 36.272999999999996 type value map_at_5 39.172000000000004 type value mrr_at_1 29.634994206257243 type value mrr_at_10 43.814949695598514 type value mrr_at_100 44.75158330890793 type value mrr_at_1000 44.76933611785972 type value mrr_at_20 44.450136580422104 type value mrr_at_3 39.56160679799143 type value mrr_at_5 42.083333333333336 type value nauc_map_at_1000_diff1 31.377733390159623 type value nauc_map_at_1000_max 10.852802240297759 type value nauc_map_at_1000_std -8.156368414989963 type value nauc_map_at_100_diff1 31.37926107010834 type value nauc_map_at_100_max 10.866567017386616 type value nauc_map_at_100_std -8.13083658675661 type value nauc_map_at_10_diff1 31.302395420970413 type value nauc_map_at_10_max 10.696471249499485 type value nauc_map_at_10_std -8.608828614048587 type value nauc_map_at_1_diff1 34.515378947817545 type value nauc_map_at_1_max 8.23278785130009 type value nauc_map_at_1_std -8.790135666737623 type value nauc_map_at_20_diff1 31.405784027747636 type value nauc_map_at_20_max 10.743222784357599 type value nauc_map_at_20_std -8.336520716356294 type value nauc_map_at_3_diff1 30.790756885918242 type value nauc_map_at_3_max 9.611996527156451 type value nauc_map_at_3_std -10.30419579409286 type value nauc_map_at_5_diff1 31.018701056437692 type value nauc_map_at_5_max 10.415471498676181 type value nauc_map_at_5_std -9.267868426130615 type value nauc_mrr_at_1000_diff1 30.954103753005274 type value nauc_mrr_at_1000_max 11.65610034595576 type value nauc_mrr_at_1000_std -6.236607914879178 type value nauc_mrr_at_100_diff1 30.95419964742793 type value nauc_mrr_at_100_max 11.67074501272962 type value nauc_mrr_at_100_std -6.2148004414091504 type value nauc_mrr_at_10_diff1 30.909871849241917 type value nauc_mrr_at_10_max 11.663150347843652 type value nauc_mrr_at_10_std -6.412145873320221 type value nauc_mrr_at_1_diff1 33.69803436461973 type value nauc_mrr_at_1_max 9.810616582626253 type value nauc_mrr_at_1_std -6.5168183653335845 type value nauc_mrr_at_20_diff1 30.97036659208301 type value nauc_mrr_at_20_max 11.615291040042264 type value nauc_mrr_at_20_std -6.317206649176624 type value nauc_mrr_at_3_diff1 30.347687412668307 type value nauc_mrr_at_3_max 11.045997984562728 type value nauc_mrr_at_3_std -7.344237528386735 type value nauc_mrr_at_5_diff1 30.607591550974323 type value nauc_mrr_at_5_max 11.478687020349025 type value nauc_mrr_at_5_std -6.773130489910162 type value nauc_ndcg_at_1000_diff1 30.721715941822435 type value nauc_ndcg_at_1000_max 12.363613568822352 type value nauc_ndcg_at_1000_std -6.083916245339269 type value nauc_ndcg_at_100_diff1 30.608831858292408 type value nauc_ndcg_at_100_max 12.894646588979683 type value nauc_ndcg_at_100_std -5.148801091143074 type value nauc_ndcg_at_10_diff1 30.483771661792847 type value nauc_ndcg_at_10_max 12.18129035771911 type value nauc_ndcg_at_10_std -7.165744970217042 type value nauc_ndcg_at_1_diff1 33.79845141868468 type value nauc_ndcg_at_1_max 9.88864563426806 type value nauc_ndcg_at_1_std -6.43552016535101 type value nauc_ndcg_at_20_diff1 30.77504113488907 type value nauc_ndcg_at_20_max 12.28245448589153 type value nauc_ndcg_at_20_std -6.325276590452571 type value nauc_ndcg_at_3_diff1 29.602918057743278 type value nauc_ndcg_at_3_max 10.39055264754259 type value nauc_ndcg_at_3_std -10.014843769784985 type value nauc_ndcg_at_5_diff1 29.94463296702168 type value nauc_ndcg_at_5_max 11.551920125900473 type value nauc_ndcg_at_5_std -8.48593988495145 type value nauc_precision_at_1000_diff1 -5.690546724212895 type value nauc_precision_at_1000_max 9.109366247129207 type value nauc_precision_at_1000_std 14.65465630262207 type value nauc_precision_at_100_diff1 -1.2336613199255233 type value nauc_precision_at_100_max 14.632255993612098 type value nauc_precision_at_100_std 20.106751006299508 type value nauc_precision_at_10_diff1 16.156638161044377 type value nauc_precision_at_10_max 15.461271728023455 type value nauc_precision_at_10_std 4.613330902566019 type value nauc_precision_at_1_diff1 33.79845141868468 type value nauc_precision_at_1_max 9.88864563426806 type value nauc_precision_at_1_std -6.43552016535101 type value nauc_precision_at_20_diff1 10.833258836740004 type value nauc_precision_at_20_max 14.399547246551503 type value nauc_precision_at_20_std 10.691750912308304 type value nauc_precision_at_3_diff1 23.440967729505452 type value nauc_precision_at_3_max 12.708378101618688 type value nauc_precision_at_3_std -7.2002199170375105 type value nauc_precision_at_5_diff1 20.632161061662867 type value nauc_precision_at_5_max 14.803138265646187 type value nauc_precision_at_5_std -1.9170585171231866 type value nauc_recall_at_1000_diff1 17.469814268756277 type value nauc_recall_at_1000_max 67.91132861575576 type value nauc_recall_at_1000_std 59.719785001643054 type value nauc_recall_at_100_diff1 20.871489158949146 type value nauc_recall_at_100_max 42.25616221901811 type value nauc_recall_at_100_std 41.83257983711543 type value nauc_recall_at_10_diff1 26.116159187824273 type value nauc_recall_at_10_max 15.673928195577544 type value nauc_recall_at_10_std -4.068034337550412 type value nauc_recall_at_1_diff1 34.515378947817545 type value nauc_recall_at_1_max 8.23278785130009 type value nauc_recall_at_1_std -8.790135666737623 type value nauc_recall_at_20_diff1 26.830515495608314 type value nauc_recall_at_20_max 17.956121895077352 type value nauc_recall_at_20_std 1.8149755315374414 type value nauc_recall_at_3_diff1 25.57777694351554 type value nauc_recall_at_3_max 10.768605841163243 type value nauc_recall_at_3_std -11.548054988544685 type value nauc_recall_at_5_diff1 25.69071002325843 type value nauc_recall_at_5_max 13.248151375739594 type value nauc_recall_at_5_std -8.31127808515032 type value ndcg_at_1 29.605999999999998 type value ndcg_at_10 49.539 type value ndcg_at_100 54.67999999999999 type value ndcg_at_1000 55.287 type value ndcg_at_20 52.196 type value ndcg_at_3 40.111999999999995 type value ndcg_at_5 44.983000000000004 type value precision_at_1 29.605999999999998 type value precision_at_10 8.607 type value precision_at_100 1.147 type value precision_at_1000 0.121 type value precision_at_20 4.938 type value precision_at_3 18.627 type value precision_at_5 13.927999999999999 type value recall_at_1 26.016000000000002 type value recall_at_10 72.51100000000001 type value recall_at_100 94.60499999999999 type value recall_at_1000 99.054 type value recall_at_20 82.353 type value recall_at_3 47.989 type value recall_at_5 59.243 type Retrieval dataset metrics task config name revision split type default MTEB QuoraRetrieval e4e08e0b7dbe3c8700f0daef558ff32256715259 test mteb/quora type value main_score 89.387 type value map_at_1 71.61699999999999 type value map_at_10 85.785 type value map_at_100 86.407 type value map_at_1000 86.42 type value map_at_20 86.206 type value map_at_3 82.867 type value map_at_5 84.736 type value mrr_at_1 82.49 type value mrr_at_10 88.59147619047603 type value mrr_at_100 88.67100295673903 type value mrr_at_1000 88.67132516200078 type value mrr_at_20 88.6561804240649 type value mrr_at_3 87.72499999999982 type value mrr_at_5 88.34599999999975 type value nauc_map_at_1000_diff1 77.75322227698767 type value nauc_map_at_1000_max 27.15325474904755 type value nauc_map_at_1000_std -45.950703261401266 type value nauc_map_at_100_diff1 77.75046471198675 type value nauc_map_at_100_max 27.125684918574887 type value nauc_map_at_100_std -46.00793046653974 type value nauc_map_at_10_diff1 77.96301805869726 type value nauc_map_at_10_max 26.63787475984541 type value nauc_map_at_10_std -48.2092244990593 type value nauc_map_at_1_diff1 81.04847175933422 type value nauc_map_at_1_max 20.828021860691376 type value nauc_map_at_1_std -40.4427741623345 type value nauc_map_at_20_diff1 77.82691021180123 type value nauc_map_at_20_max 26.979439675350086 type value nauc_map_at_20_std -46.94206477224242 type value nauc_map_at_3_diff1 78.57251235300281 type value nauc_map_at_3_max 24.306776325229592 type value nauc_map_at_3_std -50.446232609379706 type value nauc_map_at_5_diff1 78.23538738312993 type value nauc_map_at_5_max 26.005150155221003 type value nauc_map_at_5_std -49.72081450369548 type value nauc_mrr_at_1000_diff1 78.29655431237718 type value nauc_mrr_at_1000_max 29.392496550114718 type value nauc_mrr_at_1000_std -41.08607589889516 type value nauc_mrr_at_100_diff1 78.29662146607758 type value nauc_mrr_at_100_max 29.393300424020218 type value nauc_mrr_at_100_std -41.086465937239026 type value nauc_mrr_at_10_diff1 78.30206302797494 type value nauc_mrr_at_10_max 29.367617601691403 type value nauc_mrr_at_10_std -41.241804159667225 type value nauc_mrr_at_1_diff1 79.00375724290345 type value nauc_mrr_at_1_max 29.763227602149133 type value nauc_mrr_at_1_std -37.58361433096388 type value nauc_mrr_at_20_diff1 78.29875275029173 type value nauc_mrr_at_20_max 29.39463895371502 type value nauc_mrr_at_20_std -41.13808938179999 type value nauc_mrr_at_3_diff1 78.04981713424701 type value nauc_mrr_at_3_max 28.760448174610858 type value nauc_mrr_at_3_std -42.25770370267669 type value nauc_mrr_at_5_diff1 78.24030781659526 type value nauc_mrr_at_5_max 29.4627965404159 type value nauc_mrr_at_5_std -41.48382971161236 type value nauc_ndcg_at_1000_diff1 77.63586978346414 type value nauc_ndcg_at_1000_max 28.36041361858413 type value nauc_ndcg_at_1000_std -43.84956631664592 type value nauc_ndcg_at_100_diff1 77.5782899412669 type value nauc_ndcg_at_100_max 28.175349147299023 type value nauc_ndcg_at_100_std -44.03384730985532 type value nauc_ndcg_at_10_diff1 77.65612732311726 type value nauc_ndcg_at_10_max 27.447934213310145 type value nauc_ndcg_at_10_std -47.477846933136206 type value nauc_ndcg_at_1_diff1 79.00375724290345 type value nauc_ndcg_at_1_max 29.763227602149133 type value nauc_ndcg_at_1_std -37.58361433096388 type value nauc_ndcg_at_20_diff1 77.6857905925127 type value nauc_ndcg_at_20_max 27.85965135690326 type value nauc_ndcg_at_20_std -46.035623659567534 type value nauc_ndcg_at_3_diff1 77.20000663124452 type value nauc_ndcg_at_3_max 25.83926946771269 type value nauc_ndcg_at_3_std -48.46047480037077 type value nauc_ndcg_at_5_diff1 77.47304156996891 type value nauc_ndcg_at_5_max 27.277217473255703 type value nauc_ndcg_at_5_std -48.29036456924513 type value nauc_precision_at_1000_diff1 -44.34289619168728 type value nauc_precision_at_1000_max -3.3267888861609882 type value nauc_precision_at_1000_std 40.7640626789122 type value nauc_precision_at_100_diff1 -44.40180123691582 type value nauc_precision_at_100_max -4.036815279824888 type value nauc_precision_at_100_std 40.258738157948144 type value nauc_precision_at_10_diff1 -40.174969736392725 type value nauc_precision_at_10_max -1.2107921107014503 type value nauc_precision_at_10_std 26.914317558152383 type value nauc_precision_at_1_diff1 79.00375724290345 type value nauc_precision_at_1_max 29.763227602149133 type value nauc_precision_at_1_std -37.58361433096388 type value nauc_precision_at_20_diff1 -42.997551532370395 type value nauc_precision_at_20_max -2.7260912846581435 type value nauc_precision_at_20_std 33.47494527610656 type value nauc_precision_at_3_diff1 -21.172181060238913 type value nauc_precision_at_3_max 4.5591660958836835 type value nauc_precision_at_3_std 4.474651862429931 type value nauc_precision_at_5_diff1 -33.376618015297154 type value nauc_precision_at_5_max 1.7302644290575764 type value nauc_precision_at_5_std 16.980633045220895 type value nauc_recall_at_1000_diff1 58.24743045343488 type value nauc_recall_at_1000_max -21.258859048904625 type value nauc_recall_at_1000_std 5.841590725271873 type value nauc_recall_at_100_diff1 64.62432244425025 type value nauc_recall_at_100_max 11.438889005688548 type value nauc_recall_at_100_std -48.21565456849923 type value nauc_recall_at_10_diff1 73.84516212868728 type value nauc_recall_at_10_max 21.581336143130912 type value nauc_recall_at_10_std -71.40446430175044 type value nauc_recall_at_1_diff1 81.04847175933422 type value nauc_recall_at_1_max 20.828021860691376 type value nauc_recall_at_1_std -40.4427741623345 type value nauc_recall_at_20_diff1 74.07490425440125 type value nauc_recall_at_20_max 22.741699258253938 type value nauc_recall_at_20_std -75.22910750948694 type value nauc_recall_at_3_diff1 74.81258758793922 type value nauc_recall_at_3_max 19.256464797371688 type value nauc_recall_at_3_std -61.27309744783545 type value nauc_recall_at_5_diff1 73.49570838483187 type value nauc_recall_at_5_max 22.485129670655922 type value nauc_recall_at_5_std -64.95541946081566 type value ndcg_at_1 82.49 type value ndcg_at_10 89.387 type value ndcg_at_100 90.464 type value ndcg_at_1000 90.533 type value ndcg_at_20 90.01599999999999 type value ndcg_at_3 86.726 type value ndcg_at_5 88.249 type value precision_at_1 82.49 type value precision_at_10 13.543 type value precision_at_100 1.5350000000000001 type value precision_at_1000 0.157 type value precision_at_20 7.185 type value precision_at_3 37.983 type value precision_at_5 24.954 type value recall_at_1 71.61699999999999 type value recall_at_10 96.207 type value recall_at_100 99.726 type value recall_at_1000 99.991 type value recall_at_20 98.188 type value recall_at_3 88.466 type value recall_at_5 92.83200000000001 type Retrieval dataset metrics task config name revision split type default MTEB RedditClustering 24640382cdbf8abc73003fb0fa6d111a705499eb test mteb/reddit-clustering type value main_score 55.165421795067815 type value v_measure 55.165421795067815 type value v_measure_std 4.407201142010862 type Clustering dataset metrics task config name revision split type default MTEB RedditClusteringP2P 385e3cb46b4cfa89021f56c4380204149d0efe33 test mteb/reddit-clustering-p2p type value main_score 64.40104113698271 type value v_measure 64.40104113698271 type value v_measure_std 13.302523246335362 type Clustering dataset metrics task config name revision split type default MTEB SCIDOCS f8c2fcf00f625baaa80f62ec5bd9e1fff3b8ae88 test mteb/scidocs type value main_score 20.429 type value map_at_1 4.868 type value map_at_10 12.27 type value map_at_100 14.332 type value map_at_1000 14.625 type value map_at_20 13.333 type value map_at_3 8.795 type value map_at_5 10.392 type value mrr_at_1 24.0 type value mrr_at_10 34.65333333333329 type value mrr_at_100 35.674251079833766 type value mrr_at_1000 35.73520785942911 type value mrr_at_20 35.22774876654128 type value mrr_at_3 31.166666666666664 type value mrr_at_5 33.281666666666624 type value nauc_map_at_1000_diff1 17.399043123319522 type value nauc_map_at_1000_max 31.2734183775543 type value nauc_map_at_1000_std 17.077403711100832 type value nauc_map_at_100_diff1 17.403713887640865 type value nauc_map_at_100_max 31.27377201272501 type value nauc_map_at_100_std 16.87360366282937 type value nauc_map_at_10_diff1 17.359001538120168 type value nauc_map_at_10_max 30.468920168811948 type value nauc_map_at_10_std 13.380268231544715 type value nauc_map_at_1_diff1 21.421764472532455 type value nauc_map_at_1_max 22.406495947870948 type value nauc_map_at_1_std 7.278461750059741 type value nauc_map_at_20_diff1 17.309681501618616 type value nauc_map_at_20_max 30.723309484933736 type value nauc_map_at_20_std 15.103661234366466 type value nauc_map_at_3_diff1 19.21373088647576 type value nauc_map_at_3_max 28.20473469906757 type value nauc_map_at_3_std 8.112728025403056 type value nauc_map_at_5_diff1 18.058060387271972 type value nauc_map_at_5_max 30.126841947570814 type value nauc_map_at_5_std 10.52754125285907 type value nauc_mrr_at_1000_diff1 19.441702934302622 type value nauc_mrr_at_1000_max 25.596393086654306 type value nauc_mrr_at_1000_std 12.03335655261492 type value nauc_mrr_at_100_diff1 19.45550504725835 type value nauc_mrr_at_100_max 25.616075945406113 type value nauc_mrr_at_100_std 12.064272002353919 type value nauc_mrr_at_10_diff1 19.439283557585867 type value nauc_mrr_at_10_max 25.630347604493288 type value nauc_mrr_at_10_std 12.031032042077703 type value nauc_mrr_at_1_diff1 21.522585669781943 type value nauc_mrr_at_1_max 22.47948118859334 type value nauc_mrr_at_1_std 7.382278936017263 type value nauc_mrr_at_20_diff1 19.41398208318509 type value nauc_mrr_at_20_max 25.627882587061446 type value nauc_mrr_at_20_std 12.073194157092846 type value nauc_mrr_at_3_diff1 19.605200019472257 type value nauc_mrr_at_3_max 25.325244620209876 type value nauc_mrr_at_3_std 9.621890524197736 type value nauc_mrr_at_5_diff1 19.39540169944071 type value nauc_mrr_at_5_max 25.603584740156034 type value nauc_mrr_at_5_std 11.176904475558963 type value nauc_ndcg_at_1000_diff1 16.677472512130397 type value nauc_ndcg_at_1000_max 30.803531883263386 type value nauc_ndcg_at_1000_std 24.271183062150264 type value nauc_ndcg_at_100_diff1 17.36630862763037 type value nauc_ndcg_at_100_max 31.94802140143363 type value nauc_ndcg_at_100_std 23.50492571448407 type value nauc_ndcg_at_10_diff1 16.96591943739385 type value nauc_ndcg_at_10_max 29.983229462186355 type value nauc_ndcg_at_10_std 16.195748077489096 type value nauc_ndcg_at_1_diff1 21.522585669781943 type value nauc_ndcg_at_1_max 22.47948118859334 type value nauc_ndcg_at_1_std 7.382278936017263 type value nauc_ndcg_at_20_diff1 16.95752397256498 type value nauc_ndcg_at_20_max 30.17083071239411 type value nauc_ndcg_at_20_std 18.58280825082001 type value nauc_ndcg_at_3_diff1 18.84612108439313 type value nauc_ndcg_at_3_max 27.98191818651593 type value nauc_ndcg_at_3_std 9.424277024329921 type value nauc_ndcg_at_5_diff1 17.508065912086675 type value nauc_ndcg_at_5_max 29.611412732203608 type value nauc_ndcg_at_5_std 12.623793734445126 type value nauc_precision_at_1000_diff1 6.265199779097322 type value nauc_precision_at_1000_max 20.008066463216657 type value nauc_precision_at_1000_std 35.98021866405677 type value nauc_precision_at_100_diff1 11.877723135952802 type value nauc_precision_at_100_max 28.979530033834557 type value nauc_precision_at_100_std 33.61448120665875 type value nauc_precision_at_10_diff1 13.347374773447774 type value nauc_precision_at_10_max 29.532781336663056 type value nauc_precision_at_10_std 20.58195880074721 type value nauc_precision_at_1_diff1 21.522585669781943 type value nauc_precision_at_1_max 22.47948118859334 type value nauc_precision_at_1_std 7.382278936017263 type value nauc_precision_at_20_diff1 12.623490622184555 type value nauc_precision_at_20_max 27.985132320790147 type value nauc_precision_at_20_std 24.017624920206707 type value nauc_precision_at_3_diff1 17.586564287642346 type value nauc_precision_at_3_max 30.03148650786217 type value nauc_precision_at_3_std 10.379451374554094 type value nauc_precision_at_5_diff1 14.824891223085926 type value nauc_precision_at_5_max 31.410239486293527 type value nauc_precision_at_5_std 15.624402346760954 type value nauc_recall_at_1000_diff1 6.310837044332995 type value nauc_recall_at_1000_max 20.095529403256776 type value nauc_recall_at_1000_std 36.54872612878018 type value nauc_recall_at_100_diff1 12.038563848928966 type value nauc_recall_at_100_max 28.986817020127525 type value nauc_recall_at_100_std 33.54721716249713 type value nauc_recall_at_10_diff1 13.26933896316366 type value nauc_recall_at_10_max 29.38186602785486 type value nauc_recall_at_10_std 20.275621953504526 type value nauc_recall_at_1_diff1 21.421764472532455 type value nauc_recall_at_1_max 22.406495947870948 type value nauc_recall_at_1_std 7.278461750059741 type value nauc_recall_at_20_diff1 12.570312459960123 type value nauc_recall_at_20_max 27.709620758158497 type value nauc_recall_at_20_std 23.607200666051515 type value nauc_recall_at_3_diff1 17.403838471827413 type value nauc_recall_at_3_max 30.03567479942994 type value nauc_recall_at_3_std 10.168877039526405 type value nauc_recall_at_5_diff1 14.617283448905278 type value nauc_recall_at_5_max 31.260794318671316 type value nauc_recall_at_5_std 15.292480271424239 type value ndcg_at_1 24.0 type value ndcg_at_10 20.429 type value ndcg_at_100 28.327999999999996 type value ndcg_at_1000 33.489999999999995 type value ndcg_at_20 23.236 type value ndcg_at_3 19.36 type value ndcg_at_5 16.866 type value precision_at_1 24.0 type value precision_at_10 10.58 type value precision_at_100 2.196 type value precision_at_1000 0.344 type value precision_at_20 6.9 type value precision_at_3 17.967 type value precision_at_5 14.74 type value recall_at_1 4.868 type value recall_at_10 21.47 type value recall_at_100 44.622 type value recall_at_1000 69.777 type value recall_at_20 28.028 type value recall_at_3 10.933 type value recall_at_5 14.948 type Retrieval dataset metrics task config name revision split type default MTEB SICK-R 20a6d6f312dd54037fe07a32d58e5e168867909d test mteb/sickr-sts type value cosine_pearson 83.56937382314794 type value cosine_spearman 79.63245426461405 type value euclidean_pearson 81.23038281326936 type value euclidean_spearman 79.63246287500021 type value main_score 79.63245426461405 type value manhattan_pearson 81.22715334724163 type value manhattan_spearman 79.47235517811446 type value pearson 83.56937382314794 type value spearman 79.63245426461405 type STS dataset metrics task config name revision split type default MTEB STS12 a0d554a64d88156834ff5ae9920b964011b16384 test mteb/sts12-sts type value cosine_pearson 87.94074172378106 type value cosine_spearman 81.49535893255212 type value euclidean_pearson 85.67127466141365 type value euclidean_spearman 81.49519105826656 type value main_score 81.49535893255212 type value manhattan_pearson 85.7939378777207 type value manhattan_spearman 81.68788285150019 type value pearson 87.94074172378106 type value spearman 81.49535893255212 type STS dataset metrics task config name revision split type default MTEB STS13 7e90230a92c190f1bf69ae9002b8cea547a64cca test mteb/sts13-sts type value cosine_pearson 83.13868249088958 type value cosine_spearman 84.49255715794354 type value euclidean_pearson 83.94702761019037 type value euclidean_spearman 84.49261181536836 type value main_score 84.49255715794354 type value manhattan_pearson 84.05461037469608 type value manhattan_spearman 84.58504951653568 type value pearson 83.13868249088958 type value spearman 84.49255715794354 type STS dataset metrics task config name revision split type default MTEB STS14 6031580fec1f6af667f0bd2da0a551cf4f0b2375 test mteb/sts14-sts type value cosine_pearson 80.86639951141099 type value cosine_spearman 80.05601661201852 type value euclidean_pearson 80.97495767233256 type value euclidean_spearman 80.05600716279979 type value main_score 80.05601661201852 type value manhattan_pearson 80.68673997093622 type value manhattan_spearman 79.895855702411 type value pearson 80.86639951141099 type value spearman 80.05601661201852 type STS dataset metrics task config name revision split type default MTEB STS15 ae752c7c21bf194d8b67fd573edf7ae58183cbe3 test mteb/sts15-sts type value cosine_pearson 84.13791770600066 type value cosine_spearman 86.54345663501209 type value euclidean_pearson 85.62978165451675 type value euclidean_spearman 86.54346234593214 type value main_score 86.54345663501209 type value manhattan_pearson 85.3032964455555 type value manhattan_spearman 86.30088652823572 type value pearson 84.13791770600066 type value spearman 86.54345663501209 type STS dataset metrics task config name revision split type default MTEB STS16 4d8694f8f0e0100860b497b999b3dbed754a0513 test mteb/sts16-sts type value cosine_pearson 84.40315982722548 type value cosine_spearman 85.40751435377788 type value euclidean_pearson 84.35271010578505 type value euclidean_spearman 85.40751373941698 type value main_score 85.40751435377788 type value manhattan_pearson 84.17785174793401 type value manhattan_spearman 85.23156904732424 type value pearson 84.40315982722548 type value spearman 85.40751435377788 type STS dataset metrics task config name revision split type en-ar MTEB STS17 (en-ar) faeb762787bd10488a50c8b5be4a3b82e411949c test mteb/sts17-crosslingual-sts type value cosine_pearson 59.98924365555529 type value cosine_spearman 60.12821686053337 type value euclidean_pearson 60.90431312863765 type value euclidean_spearman 60.12821686053337 type value main_score 60.12821686053337 type value manhattan_pearson 59.05369093717122 type value manhattan_spearman 57.65837693471568 type value pearson 59.98924365555529 type value spearman 60.12821686053337 type STS dataset metrics task config name revision split type en-de MTEB STS17 (en-de) faeb762787bd10488a50c8b5be4a3b82e411949c test mteb/sts17-crosslingual-sts type value cosine_pearson 74.95271349225828 type value cosine_spearman 75.43839974308261 type value euclidean_pearson 75.68179466828151 type value euclidean_spearman 75.43839974308261 type value main_score 75.43839974308261 type value manhattan_pearson 75.4848070012919 type value manhattan_spearman 74.92507658877852 type value pearson 74.95271349225828 type value spearman 75.43839974308261 type STS dataset metrics task config name revision split type en-en MTEB STS17 (en-en) faeb762787bd10488a50c8b5be4a3b82e411949c test mteb/sts17-crosslingual-sts type value cosine_pearson 86.18555151297676 type value cosine_spearman 86.40304228488033 type value euclidean_pearson 86.8788548303146 type value euclidean_spearman 86.40304228488033 type value main_score 86.40304228488033 type value manhattan_pearson 86.79312171236047 type value manhattan_spearman 86.26008520753594 type value pearson 86.18555151297676 type value spearman 86.40304228488033 type STS dataset metrics task config name revision split type en-tr MTEB STS17 (en-tr) faeb762787bd10488a50c8b5be4a3b82e411949c test mteb/sts17-crosslingual-sts type value cosine_pearson 54.99479996647493 type value cosine_spearman 53.67766339389046 type value euclidean_pearson 55.32473081178422 type value euclidean_spearman 53.67766339389046 type value main_score 53.67766339389046 type value manhattan_pearson 54.66604584985125 type value manhattan_spearman 52.48322788533404 type value pearson 54.99479996647493 type value spearman 53.67766339389046 type STS dataset metrics task config name revision split type es-en MTEB STS17 (es-en) faeb762787bd10488a50c8b5be4a3b82e411949c test mteb/sts17-crosslingual-sts type value cosine_pearson 76.65184590191937 type value cosine_spearman 78.04569100389011 type value euclidean_pearson 77.11425698246029 type value euclidean_spearman 78.04569100389011 type value main_score 78.04569100389011 type value manhattan_pearson 77.34799982307821 type value manhattan_spearman 78.22975685912238 type value pearson 76.65184590191937 type value spearman 78.04569100389011 type STS dataset metrics task config name revision split type fr-en MTEB STS17 (fr-en) faeb762787bd10488a50c8b5be4a3b82e411949c test mteb/sts17-crosslingual-sts type value cosine_pearson 76.30743924244035 type value cosine_spearman 75.2110676227775 type value euclidean_pearson 77.10837892816058 type value euclidean_spearman 75.2110676227775 type value main_score 75.2110676227775 type value manhattan_pearson 76.814009334774 type value manhattan_spearman 74.96159426113054 type value pearson 76.30743924244035 type value spearman 75.2110676227775 type STS dataset metrics task config name revision split type it-en MTEB STS17 (it-en) faeb762787bd10488a50c8b5be4a3b82e411949c test mteb/sts17-crosslingual-sts type value cosine_pearson 75.11771819741416 type value cosine_spearman 74.96778304560281 type value euclidean_pearson 75.56941540554674 type value euclidean_spearman 74.96778304560281 type value main_score 74.96778304560281 type value manhattan_pearson 75.18422319871718 type value manhattan_spearman 74.45788102060328 type value pearson 75.11771819741416 type value spearman 74.96778304560281 type STS dataset metrics task config name revision split type nl-en MTEB STS17 (nl-en) faeb762787bd10488a50c8b5be4a3b82e411949c test mteb/sts17-crosslingual-sts type value cosine_pearson 72.42454093118816 type value cosine_spearman 71.9097547231894 type value euclidean_pearson 73.04051728705643 type value euclidean_spearman 71.9097547231894 type value main_score 71.9097547231894 type value manhattan_pearson 72.5487755597775 type value manhattan_spearman 71.080265405627 type value pearson 72.42454093118816 type value spearman 71.9097547231894 type STS dataset metrics task config name revision split type en MTEB STS22 (en) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 69.3881685924264 type value cosine_spearman 69.37162939123382 type value euclidean_pearson 70.5377770359738 type value euclidean_spearman 69.37162939123382 type value main_score 69.37162939123382 type value manhattan_pearson 70.86501303890763 type value manhattan_spearman 69.54018077011284 type value pearson 69.3881685924264 type value spearman 69.37162939123382 type STS dataset metrics task config name revision split type de-en MTEB STS22 (de-en) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 64.64985744446284 type value cosine_spearman 63.89323074678119 type value euclidean_pearson 66.9623010036117 type value euclidean_spearman 63.89323074678119 type value main_score 63.89323074678119 type value manhattan_pearson 68.60076281156398 type value manhattan_spearman 64.80183430943912 type value pearson 64.64985744446284 type value spearman 63.89323074678119 type STS dataset metrics task config name revision split type es-en MTEB STS22 (es-en) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 78.95094282575697 type value cosine_spearman 80.66341954222823 type value euclidean_pearson 79.7677956183949 type value euclidean_spearman 80.66341954222823 type value main_score 80.66341954222823 type value manhattan_pearson 81.52201735972797 type value manhattan_spearman 81.65309541429473 type value pearson 78.95094282575697 type value spearman 80.66341954222823 type STS dataset metrics task config name revision split type pl-en MTEB STS22 (pl-en) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 77.99167158750629 type value cosine_spearman 77.00326330683939 type value euclidean_pearson 77.60571751826936 type value euclidean_spearman 77.00326330683939 type value main_score 77.00326330683939 type value manhattan_pearson 78.19839585217989 type value manhattan_spearman 78.44894390841364 type value pearson 77.99167158750629 type value spearman 77.00326330683939 type STS dataset metrics task config name revision split type zh-en MTEB STS22 (zh-en) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 77.21035942564082 type value cosine_spearman 76.57212143103963 type value euclidean_pearson 78.03973868360728 type value euclidean_spearman 76.57212143103963 type value main_score 76.57212143103963 type value manhattan_pearson 78.16591898142042 type value manhattan_spearman 76.83958214147293 type value pearson 77.21035942564082 type value spearman 76.57212143103963 type STS dataset metrics task config name revision split type default MTEB STSBenchmark b0fddb56ed78048fa8b90373c8a3cfc37b684831 test mteb/stsbenchmark-sts type value cosine_pearson 81.21615375003084 type value cosine_spearman 84.2970803211202 type value euclidean_pearson 83.54765755364517 type value euclidean_spearman 84.2970803211202 type value main_score 84.2970803211202 type value manhattan_pearson 83.2769664077453 type value manhattan_spearman 84.09545601307758 type value pearson 81.21615375003084 type value spearman 84.2970803211202 type STS dataset metrics task config name revision split type default MTEB SciDocsRR d3c5e1fc0b855ab6097bf1cda04dd73947d7caab test mteb/scidocs-reranking type value main_score 80.72245608609909 type value map 80.72245608609909 type value mrr 94.86804408373035 type value nAUC_map_diff1 3.565293868431913 type value nAUC_map_max 53.87118155384518 type value nAUC_map_std 69.73850807835032 type value nAUC_mrr_diff1 48.33938058863373 type value nAUC_mrr_max 82.0796869926262 type value nAUC_mrr_std 79.20228314778093 type Reranking dataset metrics task config name revision split type default MTEB SciFact 0228b52cf27578f30900b9e5271d331663a030d7 test mteb/scifact type value main_score 72.604 type value map_at_1 57.05 type value map_at_10 68.026 type value map_at_100 68.54299999999999 type value map_at_1000 68.56 type value map_at_20 68.329 type value map_at_3 65.565 type value map_at_5 66.81899999999999 type value mrr_at_1 60.0 type value mrr_at_10 68.97116402116401 type value mrr_at_100 69.43171438050388 type value mrr_at_1000 69.44900642374887 type value mrr_at_20 69.25799802049801 type value mrr_at_3 67.11111111111111 type value mrr_at_5 68.27777777777779 type value nauc_map_at_1000_diff1 66.45098144160822 type value nauc_map_at_1000_max 52.26713946112144 type value nauc_map_at_1000_std -3.2435941711161194 type value nauc_map_at_100_diff1 66.45069255591629 type value nauc_map_at_100_max 52.277529223166994 type value nauc_map_at_100_std -3.236289003540743 type value nauc_map_at_10_diff1 66.50847900934123 type value nauc_map_at_10_max 52.56336813799116 type value nauc_map_at_10_std -3.2225840417202547 type value nauc_map_at_1_diff1 69.8066007922827 type value nauc_map_at_1_max 46.19700236373352 type value nauc_map_at_1_std -11.167127232139137 type value nauc_map_at_20_diff1 66.49775686319742 type value nauc_map_at_20_max 52.31488178119375 type value nauc_map_at_20_std -3.528866881477926 type value nauc_map_at_3_diff1 67.0124735448113 type value nauc_map_at_3_max 51.47207513467635 type value nauc_map_at_3_std -4.688170694240992 type value nauc_map_at_5_diff1 66.37338579400031 type value nauc_map_at_5_max 51.03182506884805 type value nauc_map_at_5_std -4.090110073585303 type value nauc_mrr_at_1000_diff1 66.13316468798861 type value nauc_mrr_at_1000_max 53.18661162667272 type value nauc_mrr_at_1000_std -1.1549432899803578 type value nauc_mrr_at_100_diff1 66.13308912088833 type value nauc_mrr_at_100_max 53.196523181344176 type value nauc_mrr_at_100_std -1.148961396684306 type value nauc_mrr_at_10_diff1 66.11198414850364 type value nauc_mrr_at_10_max 53.45434553493992 type value nauc_mrr_at_10_std -1.0202103385535555 type value nauc_mrr_at_1_diff1 69.18818640546156 type value nauc_mrr_at_1_max 50.224102107450285 type value nauc_mrr_at_1_std -4.4508756307510104 type value nauc_mrr_at_20_diff1 66.12038286624204 type value nauc_mrr_at_20_max 53.23900442821744 type value nauc_mrr_at_20_std -1.3453691424031584 type value nauc_mrr_at_3_diff1 66.23482655095762 type value nauc_mrr_at_3_max 53.519304370411625 type value nauc_mrr_at_3_std -1.0512555098049736 type value nauc_mrr_at_5_diff1 65.63605277411375 type value nauc_mrr_at_5_max 53.17390536531564 type value nauc_mrr_at_5_std -0.5198682324341892 type value nauc_ndcg_at_1000_diff1 65.85075826609345 type value nauc_ndcg_at_1000_max 53.814329968179045 type value nauc_ndcg_at_1000_std -0.9856729250792472 type value nauc_ndcg_at_100_diff1 65.78229528993444 type value nauc_ndcg_at_100_max 54.1747645815977 type value nauc_ndcg_at_100_std -0.47502756295876847 type value nauc_ndcg_at_10_diff1 66.00876580480991 type value nauc_ndcg_at_10_max 55.06235713538037 type value nauc_ndcg_at_10_std -1.5534145585575012 type value nauc_ndcg_at_1_diff1 69.18818640546156 type value nauc_ndcg_at_1_max 50.224102107450285 type value nauc_ndcg_at_1_std -4.4508756307510104 type value nauc_ndcg_at_20_diff1 65.95831573232856 type value nauc_ndcg_at_20_max 54.24206688010573 type value nauc_ndcg_at_20_std -2.705254164112238 type value nauc_ndcg_at_3_diff1 66.14046065126678 type value nauc_ndcg_at_3_max 54.07332075118414 type value nauc_ndcg_at_3_std -2.0119140501882793 type value nauc_ndcg_at_5_diff1 65.21102868019805 type value nauc_ndcg_at_5_max 52.596880916483165 type value nauc_ndcg_at_5_std -2.1720193236802023 type value nauc_precision_at_1000_diff1 -21.99504940846271 type value nauc_precision_at_1000_max 19.25403291298791 type value nauc_precision_at_1000_std 46.296476764054404 type value nauc_precision_at_100_diff1 -11.741691903205695 type value nauc_precision_at_100_max 25.699636707900623 type value nauc_precision_at_100_std 43.96233624765463 type value nauc_precision_at_10_diff1 11.568895847591932 type value nauc_precision_at_10_max 39.43006347212197 type value nauc_precision_at_10_std 28.751839941496836 type value nauc_precision_at_1_diff1 69.18818640546156 type value nauc_precision_at_1_max 50.224102107450285 type value nauc_precision_at_1_std -4.4508756307510104 type value nauc_precision_at_20_diff1 4.854833212085455 type value nauc_precision_at_20_max 34.19851755381116 type value nauc_precision_at_20_std 28.728626880402068 type value nauc_precision_at_3_diff1 35.04823458092479 type value nauc_precision_at_3_max 47.8670338954734 type value nauc_precision_at_3_std 19.389299130775157 type value nauc_precision_at_5_diff1 25.605002849466736 type value nauc_precision_at_5_max 43.50575999348689 type value nauc_precision_at_5_std 24.80257266140189 type value nauc_recall_at_1000_diff1 55.07703081232429 type value nauc_recall_at_1000_max 70.71661998132596 type value nauc_recall_at_1000_std 64.58916900093288 type value nauc_recall_at_100_diff1 59.97732426303837 type value nauc_recall_at_100_max 71.64532479658504 type value nauc_recall_at_100_std 37.87515006002412 type value nauc_recall_at_10_diff1 64.45621875630812 type value nauc_recall_at_10_max 64.72171592433827 type value nauc_recall_at_10_std 0.9026532647803642 type value nauc_recall_at_1_diff1 69.8066007922827 type value nauc_recall_at_1_max 46.19700236373352 type value nauc_recall_at_1_std -11.167127232139137 type value nauc_recall_at_20_diff1 63.79448821637328 type value nauc_recall_at_20_max 61.597381158568524 type value nauc_recall_at_20_std -7.27449509788767 type value nauc_recall_at_3_diff1 64.75442031192492 type value nauc_recall_at_3_max 56.12106077054382 type value nauc_recall_at_3_std -2.661587128227682 type value nauc_recall_at_5_diff1 60.82940800383688 type value nauc_recall_at_5_max 53.647222430433736 type value nauc_recall_at_5_std -0.793229884870239 type value ndcg_at_1 60.0 type value ndcg_at_10 72.604 type value ndcg_at_100 74.83800000000001 type value ndcg_at_1000 75.27199999999999 type value ndcg_at_20 73.599 type value ndcg_at_3 68.509 type value ndcg_at_5 70.352 type value precision_at_1 60.0 type value precision_at_10 9.733 type value precision_at_100 1.083 type value precision_at_1000 0.11199999999999999 type value precision_at_20 5.067 type value precision_at_3 27.444000000000003 type value precision_at_5 17.666999999999998 type value recall_at_1 57.05 type value recall_at_10 85.422 type value recall_at_100 95.333 type value recall_at_1000 98.667 type value recall_at_20 89.156 type value recall_at_3 74.211 type value recall_at_5 79.094 type Retrieval dataset metrics task config name revision split type default MTEB SprintDuplicateQuestions d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46 test mteb/sprintduplicatequestions-pairclassification type value cosine_accuracy 99.76237623762376 type value cosine_accuracy_threshold 89.08973932266235 type value cosine_ap 93.82184396471453 type value cosine_f1 87.87878787878789 type value cosine_f1_threshold 89.08973932266235 type value cosine_precision 88.77551020408163 type value cosine_recall 87.0 type value dot_accuracy 99.76237623762376 type value dot_accuracy_threshold 89.08973932266235 type value dot_ap 93.82179339271785 type value dot_f1 87.87878787878789 type value dot_f1_threshold 89.08973932266235 type value dot_precision 88.77551020408163 type value dot_recall 87.0 type value euclidean_accuracy 99.76237623762376 type value euclidean_accuracy_threshold 46.71244025230408 type value euclidean_ap 93.82184396471453 type value euclidean_f1 87.87878787878789 type value euclidean_f1_threshold 46.71244025230408 type value euclidean_precision 88.77551020408163 type value euclidean_recall 87.0 type value main_score 94.18170827750167 type value manhattan_accuracy 99.77425742574258 type value manhattan_accuracy_threshold 1095.131492614746 type value manhattan_ap 94.18170827750167 type value manhattan_f1 88.45577211394303 type value manhattan_f1_threshold 1108.85648727417 type value manhattan_precision 88.41158841158841 type value manhattan_recall 88.5 type value max_ap 94.18170827750167 type value max_f1 88.45577211394303 type value max_precision 88.77551020408163 type value max_recall 88.5 type value similarity_accuracy 99.76237623762376 type value similarity_accuracy_threshold 89.08973932266235 type value similarity_ap 93.82184396471453 type value similarity_f1 87.87878787878789 type value similarity_f1_threshold 89.08973932266235 type value similarity_precision 88.77551020408163 type value similarity_recall 87.0 type PairClassification dataset metrics task config name revision split type default MTEB StackExchangeClustering 6cbc1f7b2bc0622f2e39d2c77fa502909748c259 test mteb/stackexchange-clustering type value main_score 65.93583959980214 type value v_measure 65.93583959980214 type value v_measure_std 3.9403815544270233 type Clustering dataset metrics task config name revision split type default MTEB StackExchangeClusteringP2P 815ca46b2622cec33ccafc3735d572c266efdb44 test mteb/stackexchange-clustering-p2p type value main_score 35.594885571404724 type value v_measure 35.594885571404724 type value v_measure_std 1.5163847345337254 type Clustering dataset metrics task config name revision split type default MTEB StackOverflowDupQuestions e185fbe320c72810689fc5848eb6114e1ef5ec69 test mteb/stackoverflowdupquestions-reranking type value main_score 49.85213562933509 type value map 49.85213562933509 type value mrr 50.62702922077922 type value nAUC_map_diff1 36.55011836042864 type value nAUC_map_max 13.45991062036654 type value nAUC_map_std 10.192881915639742 type value nAUC_mrr_diff1 37.058265888016976 type value nAUC_mrr_max 14.081819232783383 type value nAUC_mrr_std 11.215978874656958 type Reranking dataset metrics task config name revision split type default MTEB SummEval cda12ad7615edc362dbf25a00fdd61d3b1eaf93c test mteb/summeval type value cosine_pearson 25.349220308622627 type value cosine_spearman 27.880975911253458 type value dot_pearson 25.349197273883224 type value dot_spearman 27.880903951553655 type value main_score 27.880975911253458 type value pearson 25.349220308622627 type value spearman 27.880975911253458 type Summarization dataset metrics task config name revision split type default MTEB TRECCOVID bb9466bac8153a0349341eb1b22e06409e78ef4e test mteb/trec-covid type value main_score 79.536 type value map_at_1 0.215 type value map_at_10 2.048 type value map_at_100 12.842999999999998 type value map_at_1000 31.032 type value map_at_20 3.8379999999999996 type value map_at_3 0.64 type value map_at_5 1.052 type value mrr_at_1 84.0 type value mrr_at_10 91.16666666666666 type value mrr_at_100 91.16666666666666 type value mrr_at_1000 91.16666666666666 type value mrr_at_20 91.16666666666666 type value mrr_at_3 90.66666666666666 type value mrr_at_5 91.16666666666666 type value nauc_map_at_1000_diff1 -18.530580290412697 type value nauc_map_at_1000_max 43.14744028154331 type value nauc_map_at_1000_std 79.6699665194256 type value nauc_map_at_100_diff1 -21.271315814062437 type value nauc_map_at_100_max 17.55081814849073 type value nauc_map_at_100_std 48.17729810787553 type value nauc_map_at_10_diff1 -11.002124943974252 type value nauc_map_at_10_max -9.6495971981689 type value nauc_map_at_10_std 6.648364965330221 type value nauc_map_at_1_diff1 0.1251393811417004 type value nauc_map_at_1_max -12.601700488498643 type value nauc_map_at_1_std -3.5018878780762366 type value nauc_map_at_20_diff1 -19.526191160714987 type value nauc_map_at_20_max -4.175483070077258 type value nauc_map_at_20_std 16.014345473073693 type value nauc_map_at_3_diff1 -0.8632406748675692 type value nauc_map_at_3_max -12.9654502212951 type value nauc_map_at_3_std -1.5551804410996426 type value nauc_map_at_5_diff1 -9.294941718115151 type value nauc_map_at_5_max -12.795655812948572 type value nauc_map_at_5_std 0.6128051906803516 type value nauc_mrr_at_1000_diff1 33.997935217447434 type value nauc_mrr_at_1000_max 41.160149696734955 type value nauc_mrr_at_1000_std 27.657024869568446 type value nauc_mrr_at_100_diff1 33.997935217447434 type value nauc_mrr_at_100_max 41.160149696734955 type value nauc_mrr_at_100_std 27.657024869568446 type value nauc_mrr_at_10_diff1 33.997935217447434 type value nauc_mrr_at_10_max 41.160149696734955 type value nauc_mrr_at_10_std 27.657024869568446 type value nauc_mrr_at_1_diff1 37.279086892488884 type value nauc_mrr_at_1_max 43.292832596956316 type value nauc_mrr_at_1_std 20.305596465390227 type value nauc_mrr_at_20_diff1 33.997935217447434 type value nauc_mrr_at_20_max 41.160149696734955 type value nauc_mrr_at_20_std 27.657024869568446 type value nauc_mrr_at_3_diff1 31.138610414926326 type value nauc_mrr_at_3_max 39.545043163464186 type value nauc_mrr_at_3_std 31.70252018936244 type value nauc_mrr_at_5_diff1 33.997935217447434 type value nauc_mrr_at_5_max 41.160149696734955 type value nauc_mrr_at_5_std 27.657024869568446 type value nauc_ndcg_at_1000_diff1 -20.948326611476556 type value nauc_ndcg_at_1000_max 36.766927406101956 type value nauc_ndcg_at_1000_std 75.32635798841658 type value nauc_ndcg_at_100_diff1 -14.54815381092273 type value nauc_ndcg_at_100_max 51.38801585344711 type value nauc_ndcg_at_100_std 76.47002281413397 type value nauc_ndcg_at_10_diff1 -12.80351464937073 type value nauc_ndcg_at_10_max 35.71831279387225 type value nauc_ndcg_at_10_std 52.15347275643156 type value nauc_ndcg_at_1_diff1 20.42160737812909 type value nauc_ndcg_at_1_max 34.20619235836624 type value nauc_ndcg_at_1_std 13.088179936005965 type value nauc_ndcg_at_20_diff1 -18.116251292365128 type value nauc_ndcg_at_20_max 46.9808896232964 type value nauc_ndcg_at_20_std 61.73761431506857 type value nauc_ndcg_at_3_diff1 -4.44558396286013 type value nauc_ndcg_at_3_max 26.953553278525938 type value nauc_ndcg_at_3_std 33.375410187254786 type value nauc_ndcg_at_5_diff1 -15.495190925371652 type value nauc_ndcg_at_5_max 29.21035888164427 type value nauc_ndcg_at_5_std 41.168078957076396 type value nauc_precision_at_1000_diff1 6.339888107354097 type value nauc_precision_at_1000_max 51.87294743895088 type value nauc_precision_at_1000_std 49.22667294372217 type value nauc_precision_at_100_diff1 -10.245901160105356 type value nauc_precision_at_100_max 56.07707608097002 type value nauc_precision_at_100_std 78.96626562096216 type value nauc_precision_at_10_diff1 -4.590219332829025 type value nauc_precision_at_10_max 47.52908614003191 type value nauc_precision_at_10_std 59.53043786106239 type value nauc_precision_at_1_diff1 37.279086892488884 type value nauc_precision_at_1_max 43.292832596956316 type value nauc_precision_at_1_std 20.305596465390227 type value nauc_precision_at_20_diff1 -14.763079024242392 type value nauc_precision_at_20_max 56.25820402898436 type value nauc_precision_at_20_std 67.6952843431086 type value nauc_precision_at_3_diff1 2.9292734630949067 type value nauc_precision_at_3_max 41.296148445888285 type value nauc_precision_at_3_std 46.551771604768255 type value nauc_precision_at_5_diff1 -15.368719472623535 type value nauc_precision_at_5_max 39.706937186186984 type value nauc_precision_at_5_std 45.991734125764275 type value nauc_recall_at_1000_diff1 -18.70157967410686 type value nauc_recall_at_1000_max 27.303031147629746 type value nauc_recall_at_1000_std 63.59247900235757 type value nauc_recall_at_100_diff1 -21.505202598262795 type value nauc_recall_at_100_max 3.1053955846040666 type value nauc_recall_at_100_std 35.59388419574821 type value nauc_recall_at_10_diff1 -13.309140466736356 type value nauc_recall_at_10_max -16.90482412154473 type value nauc_recall_at_10_std 2.1355678490728542 type value nauc_recall_at_1_diff1 0.1251393811417004 type value nauc_recall_at_1_max -12.601700488498643 type value nauc_recall_at_1_std -3.5018878780762366 type value nauc_recall_at_20_diff1 -21.303497421292096 type value nauc_recall_at_20_max -13.765429909809388 type value nauc_recall_at_20_std 9.07482009539061 type value nauc_recall_at_3_diff1 -6.017177782774693 type value nauc_recall_at_3_max -19.064966459546255 type value nauc_recall_at_3_std -3.0227410013796967 type value nauc_recall_at_5_diff1 -14.078289790672653 type value nauc_recall_at_5_max -19.52038684292809 type value nauc_recall_at_5_std -2.6267198328675994 type value ndcg_at_1 78.0 type value ndcg_at_10 79.536 type value ndcg_at_100 62.65500000000001 type value ndcg_at_1000 56.359 type value ndcg_at_20 77.561 type value ndcg_at_3 80.296 type value ndcg_at_5 79.806 type value precision_at_1 84.0 type value precision_at_10 85.6 type value precision_at_100 64.92 type value precision_at_1000 24.89 type value precision_at_20 83.2 type value precision_at_3 87.333 type value precision_at_5 87.2 type value recall_at_1 0.215 type value recall_at_10 2.246 type value recall_at_100 15.784 type value recall_at_1000 53.427 type value recall_at_20 4.281 type value recall_at_3 0.688 type value recall_at_5 1.142 type Retrieval dataset metrics task config name revision split type default MTEB Touche2020 a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f test mteb/touche2020 type value main_score 31.186999999999998 type value map_at_1 3.4070000000000005 type value map_at_10 13.313 type value map_at_100 19.900000000000002 type value map_at_1000 21.437 type value map_at_20 15.714 type value map_at_3 6.923 type value map_at_5 9.054 type value mrr_at_1 44.89795918367347 type value mrr_at_10 56.63832199546485 type value mrr_at_100 57.666166033512965 type value mrr_at_1000 57.666166033512965 type value mrr_at_20 57.51229496127455 type value mrr_at_3 53.40136054421768 type value mrr_at_5 55.1360544217687 type value nauc_map_at_1000_diff1 6.929929189103678 type value nauc_map_at_1000_max -20.5925373398606 type value nauc_map_at_1000_std 7.835669658058121 type value nauc_map_at_100_diff1 7.528899533894891 type value nauc_map_at_100_max -21.032199268806018 type value nauc_map_at_100_std 5.370650925959299 type value nauc_map_at_10_diff1 14.176770339374578 type value nauc_map_at_10_max -19.194036092916633 type value nauc_map_at_10_std -14.964801890692026 type value nauc_map_at_1_diff1 16.059944358241733 type value nauc_map_at_1_max -25.302527766801695 type value nauc_map_at_1_std -13.565207797491604 type value nauc_map_at_20_diff1 11.361043123465297 type value nauc_map_at_20_max -18.0301938420575 type value nauc_map_at_20_std -7.25573010108597 type value nauc_map_at_3_diff1 21.973707928327727 type value nauc_map_at_3_max -20.079194093834058 type value nauc_map_at_3_std -20.173080790091422 type value nauc_map_at_5_diff1 19.669071376698206 type value nauc_map_at_5_max -23.679751632414845 type value nauc_map_at_5_std -20.28001860761147 type value nauc_mrr_at_1000_diff1 6.875737447320781 type value nauc_mrr_at_1000_max -44.8769334243922 type value nauc_mrr_at_1000_std 7.361962913444513 type value nauc_mrr_at_100_diff1 6.875737447320781 type value nauc_mrr_at_100_max -44.8769334243922 type value nauc_mrr_at_100_std 7.361962913444513 type value nauc_mrr_at_10_diff1 6.574806453972689 type value nauc_mrr_at_10_max -47.267277277496596 type value nauc_mrr_at_10_std 8.783148855636174 type value nauc_mrr_at_1_diff1 12.940754496022242 type value nauc_mrr_at_1_max -35.544013626458145 type value nauc_mrr_at_1_std 6.0616339439628915 type value nauc_mrr_at_20_diff1 7.179017109424859 type value nauc_mrr_at_20_max -45.52183055340191 type value nauc_mrr_at_20_std 6.960503593984209 type value nauc_mrr_at_3_diff1 2.10431985300728 type value nauc_mrr_at_3_max -41.662819302741184 type value nauc_mrr_at_3_std 5.68448693989341 type value nauc_mrr_at_5_diff1 5.25929369032379 type value nauc_mrr_at_5_max -44.62592534259141 type value nauc_mrr_at_5_std 6.26151671868977 type value nauc_ndcg_at_1000_diff1 -6.563466320842519 type value nauc_ndcg_at_1000_max -33.15200693567147 type value nauc_ndcg_at_1000_std 29.09290649197198 type value nauc_ndcg_at_100_diff1 -4.290185637900728 type value nauc_ndcg_at_100_max -35.6991058391752 type value nauc_ndcg_at_100_std 24.47606141799262 type value nauc_ndcg_at_10_diff1 4.171305930645993 type value nauc_ndcg_at_10_max -33.02156808389195 type value nauc_ndcg_at_10_std -0.7115167969929295 type value nauc_ndcg_at_1_diff1 4.295135743080979 type value nauc_ndcg_at_1_max -30.841816609035575 type value nauc_ndcg_at_1_std 11.08702259742227 type value nauc_ndcg_at_20_diff1 5.716130418772172 type value nauc_ndcg_at_20_max -32.02017772879846 type value nauc_ndcg_at_20_std 0.42043490374547515 type value nauc_ndcg_at_3_diff1 0.7696408676847266 type value nauc_ndcg_at_3_max -28.19446012238678 type value nauc_ndcg_at_3_std 1.4270173161697919 type value nauc_ndcg_at_5_diff1 4.011877087450832 type value nauc_ndcg_at_5_max -35.474817068811866 type value nauc_ndcg_at_5_std -1.0183501951460643 type value nauc_precision_at_1000_diff1 -18.852617887278956 type value nauc_precision_at_1000_max 26.536677685298997 type value nauc_precision_at_1000_std 31.17777014427175 type value nauc_precision_at_100_diff1 -21.993356262198738 type value nauc_precision_at_100_max -14.151354806872973 type value nauc_precision_at_100_std 68.01931004336306 type value nauc_precision_at_10_diff1 3.518175306600991 type value nauc_precision_at_10_max -34.29876549408336 type value nauc_precision_at_10_std 8.571886047048881 type value nauc_precision_at_1_diff1 12.940754496022242 type value nauc_precision_at_1_max -35.544013626458145 type value nauc_precision_at_1_std 6.0616339439628915 type value nauc_precision_at_20_diff1 6.23454071647187 type value nauc_precision_at_20_max -29.16565290719762 type value nauc_precision_at_20_std 25.567483624610297 type value nauc_precision_at_3_diff1 8.77511441582519 type value nauc_precision_at_3_max -29.389312907952135 type value nauc_precision_at_3_std -6.397150206890867 type value nauc_precision_at_5_diff1 9.795445750266063 type value nauc_precision_at_5_max -38.88827845334236 type value nauc_precision_at_5_std -3.397760151003072 type value nauc_recall_at_1000_diff1 -28.033327034031043 type value nauc_recall_at_1000_max -15.30930042500693 type value nauc_recall_at_1000_std 69.27496829698434 type value nauc_recall_at_100_diff1 -12.558500592244782 type value nauc_recall_at_100_max -27.109814142314832 type value nauc_recall_at_100_std 40.23660136119213 type value nauc_recall_at_10_diff1 8.859020421080002 type value nauc_recall_at_10_max -26.101835112681034 type value nauc_recall_at_10_std -12.02508230851673 type value nauc_recall_at_1_diff1 16.059944358241733 type value nauc_recall_at_1_max -25.302527766801695 type value nauc_recall_at_1_std -13.565207797491604 type value nauc_recall_at_20_diff1 6.598503996413421 type value nauc_recall_at_20_max -25.661355219947264 type value nauc_recall_at_20_std -0.5270972932429998 type value nauc_recall_at_3_diff1 15.848752699477423 type value nauc_recall_at_3_max -20.67227958185249 type value nauc_recall_at_3_std -19.687883601951533 type value nauc_recall_at_5_diff1 15.210234895525055 type value nauc_recall_at_5_max -30.20253332454299 type value nauc_recall_at_5_std -19.986130369906242 type value ndcg_at_1 40.816 type value ndcg_at_10 31.186999999999998 type value ndcg_at_100 42.742000000000004 type value ndcg_at_1000 53.230999999999995 type value ndcg_at_20 31.057000000000002 type value ndcg_at_3 34.382000000000005 type value ndcg_at_5 32.038 type value precision_at_1 44.897999999999996 type value precision_at_10 27.143 type value precision_at_100 8.735 type value precision_at_1000 1.59 type value precision_at_20 19.898 type value precision_at_3 34.694 type value precision_at_5 31.019999999999996 type value recall_at_1 3.4070000000000005 type value recall_at_10 19.987 type value recall_at_100 52.888999999999996 type value recall_at_1000 85.172 type value recall_at_20 27.025 type value recall_at_3 7.774 type value recall_at_5 11.571 type Retrieval dataset metrics task config name revision split type default MTEB ToxicConversationsClassification edfaf9da55d3dd50d43143d90c1ac476895ae6de test mteb/toxic_conversations_50k type value accuracy 89.0380859375 type value ap 34.26536468203791 type value ap_weighted 34.26536468203791 type value f1 73.86921962038298 type value f1_weighted 90.61132302248866 type value main_score 89.0380859375 type Classification dataset metrics task config name revision split type default MTEB TweetSentimentExtractionClassification d604517c81ca91fe16a244d1248fc021f9ecee7a test mteb/tweet_sentiment_extraction type value accuracy 76.49405772495754 type value f1 76.73610452546936 type value f1_weighted 76.14362047024868 type value main_score 76.49405772495754 type Classification dataset metrics task config name revision split type default MTEB TwentyNewsgroupsClustering 6125ec4e24fa026cec8a478383ee943acfbd5449 test mteb/twentynewsgroups-clustering type value main_score 49.554702818248735 type value v_measure 49.554702818248735 type value v_measure_std 0.9278298624304031 type Clustering dataset metrics task config name revision split type default MTEB TwitterSemEval2015 70970daeab8776df92f5ea462b6173c0b46fd2d1 test mteb/twittersemeval2015-pairclassification type value cosine_accuracy 85.29534481730941 type value cosine_accuracy_threshold 90.6567394733429 type value cosine_ap 71.59976408272617 type value cosine_f1 66.54452180285818 type value cosine_f1_threshold 88.94971013069153 type value cosine_precision 61.95133045258131 type value cosine_recall 71.87335092348285 type value dot_accuracy 85.29534481730941 type value dot_accuracy_threshold 90.65674543380737 type value dot_ap 71.5997871796046 type value dot_f1 66.54452180285818 type value dot_f1_threshold 88.94971013069153 type value dot_precision 61.95133045258131 type value dot_recall 71.87335092348285 type value euclidean_accuracy 85.29534481730941 type value euclidean_accuracy_threshold 43.2279109954834 type value euclidean_ap 71.59977967634174 type value euclidean_f1 66.54452180285818 type value euclidean_f1_threshold 47.01125621795654 type value euclidean_precision 61.95133045258131 type value euclidean_recall 71.87335092348285 type value main_score 71.5997871796046 type value manhattan_accuracy 85.1820945341837 type value manhattan_accuracy_threshold 1019.9851989746094 type value manhattan_ap 71.22149639016482 type value manhattan_f1 66.31834750911301 type value manhattan_f1_threshold 1109.6149444580078 type value manhattan_precision 61.46396396396396 type value manhattan_recall 72.00527704485488 type value max_ap 71.5997871796046 type value max_f1 66.54452180285818 type value max_precision 61.95133045258131 type value max_recall 72.00527704485488 type value similarity_accuracy 85.29534481730941 type value similarity_accuracy_threshold 90.6567394733429 type value similarity_ap 71.59976408272617 type value similarity_f1 66.54452180285818 type value similarity_f1_threshold 88.94971013069153 type value similarity_precision 61.95133045258131 type value similarity_recall 71.87335092348285 type PairClassification dataset metrics task config name revision split type default MTEB TwitterURLCorpus 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf test mteb/twitterurlcorpus-pairclassification type value cosine_accuracy 89.19936352699189 type value cosine_accuracy_threshold 87.72701621055603 type value cosine_ap 86.32764736710979 type value cosine_f1 78.40269966254218 type value cosine_f1_threshold 86.80565357208252 type value cosine_precision 76.41426692004093 type value cosine_recall 80.49738219895288 type value dot_accuracy 89.19936352699189 type value dot_accuracy_threshold 87.72701621055603 type value dot_ap 86.32762879051161 type value dot_f1 78.40269966254218 type value dot_f1_threshold 86.80565357208252 type value dot_precision 76.41426692004093 type value dot_recall 80.49738219895288 type value euclidean_accuracy 89.19936352699189 type value euclidean_accuracy_threshold 49.54388439655304 type value euclidean_ap 86.3276630523782 type value euclidean_f1 78.40269966254218 type value euclidean_f1_threshold 51.36992931365967 type value euclidean_precision 76.41426692004093 type value euclidean_recall 80.49738219895288 type value main_score 86.3276630523782 type value manhattan_accuracy 89.16637559669344 type value manhattan_accuracy_threshold 1150.1700401306152 type value manhattan_ap 86.28674414277404 type value manhattan_f1 78.34183768482997 type value manhattan_f1_threshold 1213.088321685791 type value manhattan_precision 75.87475651107424 type value manhattan_recall 80.97474591931014 type value max_ap 86.3276630523782 type value max_f1 78.40269966254218 type value max_precision 76.41426692004093 type value max_recall 80.97474591931014 type value similarity_accuracy 89.19936352699189 type value similarity_accuracy_threshold 87.72701621055603 type value similarity_ap 86.32764736710979 type value similarity_f1 78.40269966254218 type value similarity_f1_threshold 86.80565357208252 type value similarity_precision 76.41426692004093 type value similarity_recall 80.49738219895288 type PairClassification dataset metrics task config name revision split type default MTEB AFQMC b44c3b011063adb25877c13823db83bb193913c4 validation C-MTEB/AFQMC type value cosine_pearson 38.59465613249044 type value cosine_spearman 39.876884773191065 type value euclidean_pearson 38.370163017159996 type value euclidean_spearman 39.87692498028858 type value main_score 39.876884773191065 type value manhattan_pearson 38.058013850119785 type value manhattan_spearman 39.531271872106856 type value pearson 38.59465613249044 type value spearman 39.876884773191065 type STS dataset metrics task config name revision split type default MTEB ATEC 0f319b1142f28d00e055a6770f3f726ae9b7d865 test C-MTEB/ATEC type value cosine_pearson 46.457799031090666 type value cosine_spearman 47.170032935367935 type value euclidean_pearson 49.399858337266004 type value euclidean_spearman 47.17003293450119 type value main_score 47.170032935367935 type value manhattan_pearson 49.19428772786887 type value manhattan_spearman 46.94649743167009 type value pearson 46.457799031090666 type value spearman 47.170032935367935 type STS dataset metrics task config name revision split type zh MTEB AmazonReviewsClassification (zh) 1399c76144fd37290681b995c656ef9b2e06e26d test mteb/amazon_reviews_multi type value accuracy 50.246 type value f1 45.84988588370862 type value f1_weighted 45.84988588370862 type value main_score 50.246 type Classification dataset metrics task config name revision split type default MTEB BQ e3dda5e115e487b39ec7e618c0c6a29137052a55 test C-MTEB/BQ type value cosine_pearson 53.67950003884396 type value cosine_spearman 54.36088598761955 type value euclidean_pearson 53.09394654913335 type value euclidean_spearman 54.36088252221325 type value main_score 54.36088598761955 type value manhattan_pearson 52.805415867146955 type value manhattan_spearman 54.06705049402532 type value pearson 53.67950003884396 type value spearman 54.36088598761955 type STS dataset metrics task config name revision split type default MTEB CLSClusteringP2P 4b6227591c6c1a73bc76b1055f3b7f3588e72476 test C-MTEB/CLSClusteringP2P type value main_score 41.608876653105966 type value v_measure 41.608876653105966 type value v_measure_std 1.0624705258546963 type Clustering dataset metrics task config name revision split type default MTEB CLSClusteringS2S e458b3f5414b62b7f9f83499ac1f5497ae2e869f test C-MTEB/CLSClusteringS2S type value main_score 39.7110966049789 type value v_measure 39.7110966049789 type value v_measure_std 0.875231943450341 type Clustering dataset metrics task config name revision split type default MTEB CMedQAv1 8d7f1e942507dac42dc58017c1a001c3717da7df test C-MTEB/CMedQAv1-reranking type value main_score 81.7193302624052 type value map 81.7193302624052 type value mrr 84.58841269841271 type value nAUC_map_diff1 57.41916975321788 type value nAUC_map_max 61.409376634272874 type value nAUC_map_std 28.913154318201233 type value nAUC_mrr_diff1 64.85350793018186 type value nAUC_mrr_max 69.46338529223004 type value nAUC_mrr_std 35.373588518165235 type Reranking dataset metrics task config name revision split type default MTEB CMedQAv2 23d186750531a14a0357ca22cd92d712fd512ea0 test C-MTEB/CMedQAv2-reranking type value main_score 82.59163356780259 type value map 82.59163356780259 type value mrr 85.54900793650792 type value nAUC_map_diff1 61.10665055831455 type value nAUC_map_max 60.91441391850925 type value nAUC_map_std 21.471788062972436 type value nAUC_mrr_diff1 69.95883630916767 type value nAUC_mrr_max 71.06959737866757 type value nAUC_mrr_std 30.819473605657606 type Reranking dataset metrics task config name revision split type default MTEB CmedqaRetrieval cd540c506dae1cf9e9a59c3e06f42030d54e7301 dev C-MTEB/CmedqaRetrieval type value main_score 42.631 type value map_at_1 24.834 type value map_at_10 36.447 type value map_at_100 38.04 type value map_at_1000 38.179 type value map_at_20 37.281 type value map_at_3 32.761 type value map_at_5 34.871 type value mrr_at_1 38.05951487871968 type value mrr_at_10 45.57554071057435 type value mrr_at_100 46.447190120013 type value mrr_at_1000 46.50606585607273 type value mrr_at_20 46.057122452003696 type value mrr_at_3 43.34000166708336 type value mrr_at_5 44.58531299491537 type value nauc_map_at_1000_diff1 48.47252945149055 type value nauc_map_at_1000_max 34.62100533042246 type value nauc_map_at_1000_std -2.684326419049642 type value nauc_map_at_100_diff1 48.43175156549248 type value nauc_map_at_100_max 34.58371253483366 type value nauc_map_at_100_std -2.719072576245476 type value nauc_map_at_10_diff1 48.18476956739444 type value nauc_map_at_10_max 33.52918292302435 type value nauc_map_at_10_std -3.746440821126843 type value nauc_map_at_1_diff1 52.68253139221022 type value nauc_map_at_1_max 26.033202075590157 type value nauc_map_at_1_std -5.756330655143574 type value nauc_map_at_20_diff1 48.33335064427594 type value nauc_map_at_20_max 34.08423189594616 type value nauc_map_at_20_std -3.2957587803371693 type value nauc_map_at_3_diff1 49.07970552101722 type value nauc_map_at_3_max 30.931354812941592 type value nauc_map_at_3_std -5.397714078300849 type value nauc_map_at_5_diff1 48.582852045037974 type value nauc_map_at_5_max 32.37350218464533 type value nauc_map_at_5_std -4.604286079722004 type value nauc_mrr_at_1000_diff1 55.36516647246729 type value nauc_mrr_at_1000_max 41.8197309169163 type value nauc_mrr_at_1000_std 1.2938880389263046 type value nauc_mrr_at_100_diff1 55.33480230365865 type value nauc_mrr_at_100_max 41.82044267368069 type value nauc_mrr_at_100_std 1.3168989639934452 type value nauc_mrr_at_10_diff1 55.25761484350501 type value nauc_mrr_at_10_max 41.625145381930565 type value nauc_mrr_at_10_std 1.0129282219497187 type value nauc_mrr_at_1_diff1 60.68654871568434 type value nauc_mrr_at_1_max 43.033167419208546 type value nauc_mrr_at_1_std 0.4003726817671297 type value nauc_mrr_at_20_diff1 55.265505678078995 type value nauc_mrr_at_20_max 41.7232926738926 type value nauc_mrr_at_20_std 1.1959474260609984 type value nauc_mrr_at_3_diff1 56.49797535079964 type value nauc_mrr_at_3_max 41.922468081636865 type value nauc_mrr_at_3_std 0.7461678066019137 type value nauc_mrr_at_5_diff1 55.726696029505305 type value nauc_mrr_at_5_max 41.7068087576993 type value nauc_mrr_at_5_std 0.9345604936396126 type value nauc_ndcg_at_1000_diff1 49.12475845061519 type value nauc_ndcg_at_1000_max 38.13450613159849 type value nauc_ndcg_at_1000_std 0.9070870161011241 type value nauc_ndcg_at_100_diff1 48.12044160559342 type value nauc_ndcg_at_100_max 37.98858612073559 type value nauc_ndcg_at_100_std 1.398027778560473 type value nauc_ndcg_at_10_diff1 47.49083707975477 type value nauc_ndcg_at_10_max 35.424124038022484 type value nauc_ndcg_at_10_std -1.9285006153671742 type value nauc_ndcg_at_1_diff1 60.68654871568434 type value nauc_ndcg_at_1_max 43.033167419208546 type value nauc_ndcg_at_1_std 0.4003726817671297 type value nauc_ndcg_at_20_diff1 47.692259910508014 type value nauc_ndcg_at_20_max 36.20333827999666 type value nauc_ndcg_at_20_std -1.1366081258269927 type value nauc_ndcg_at_3_diff1 49.926059859304004 type value nauc_ndcg_at_3_max 36.554915901584614 type value nauc_ndcg_at_3_std -1.7727717324767251 type value nauc_ndcg_at_5_diff1 48.504726113001304 type value nauc_ndcg_at_5_max 35.2222520201459 type value nauc_ndcg_at_5_std -2.1147823162180046 type value nauc_precision_at_1000_diff1 5.95771915067704 type value nauc_precision_at_1000_max 29.222734901088483 type value nauc_precision_at_1000_std 21.021319062045514 type value nauc_precision_at_100_diff1 12.441767269549631 type value nauc_precision_at_100_max 37.028610753731876 type value nauc_precision_at_100_std 22.59370573792191 type value nauc_precision_at_10_diff1 25.3055255305395 type value nauc_precision_at_10_max 41.57346735024518 type value nauc_precision_at_10_std 10.851514810119529 type value nauc_precision_at_1_diff1 60.68654871568434 type value nauc_precision_at_1_max 43.033167419208546 type value nauc_precision_at_1_std 0.4003726817671297 type value nauc_precision_at_20_diff1 21.503387725334118 type value nauc_precision_at_20_max 40.35637914704234 type value nauc_precision_at_20_std 14.15622720179941 type value nauc_precision_at_3_diff1 37.92102588120911 type value nauc_precision_at_3_max 42.61959379379323 type value nauc_precision_at_3_std 4.531204029823331 type value nauc_precision_at_5_diff1 31.822114101121624 type value nauc_precision_at_5_max 42.00621213856077 type value nauc_precision_at_5_std 7.038453918682581 type value nauc_recall_at_1000_diff1 30.906717381989445 type value nauc_recall_at_1000_max 49.86631344507457 type value nauc_recall_at_1000_std 44.77133994051694 type value nauc_recall_at_100_diff1 29.06337979940958 type value nauc_recall_at_100_max 35.64030149194558 type value nauc_recall_at_100_std 16.019430611168264 type value nauc_recall_at_10_diff1 34.92848768468913 type value nauc_recall_at_10_max 28.566945065867454 type value nauc_recall_at_10_std -2.1058035354561557 type value nauc_recall_at_1_diff1 52.68253139221022 type value nauc_recall_at_1_max 26.033202075590157 type value nauc_recall_at_1_std -5.756330655143574 type value nauc_recall_at_20_diff1 33.82932775397309 type value nauc_recall_at_20_max 29.679872190739044 type value nauc_recall_at_20_std 0.10165951410954753 type value nauc_recall_at_3_diff1 42.53700938223526 type value nauc_recall_at_3_max 27.477725171266385 type value nauc_recall_at_3_std -5.201557627828334 type value nauc_recall_at_5_diff1 39.158896850349116 type value nauc_recall_at_5_max 27.90842581577196 type value nauc_recall_at_5_std -3.646479982111823 type value ndcg_at_1 38.06 type value ndcg_at_10 42.631 type value ndcg_at_100 49.114000000000004 type value ndcg_at_1000 51.745 type value ndcg_at_20 44.895 type value ndcg_at_3 38.153999999999996 type value ndcg_at_5 39.994 type value precision_at_1 38.06 type value precision_at_10 9.35 type value precision_at_100 1.471 type value precision_at_1000 0.181 type value precision_at_20 5.461 type value precision_at_3 21.555 type value precision_at_5 15.443999999999999 type value recall_at_1 24.834 type value recall_at_10 51.881 type value recall_at_100 79.095 type value recall_at_1000 97.077 type value recall_at_20 59.471 type value recall_at_3 37.836 type value recall_at_5 43.913999999999994 type Retrieval dataset metrics task config name revision split type default MTEB Cmnli 41bc36f332156f7adc9e38f53777c959b2ae9766 validation C-MTEB/CMNLI type value cosine_accuracy 69.29645219482863 type value cosine_accuracy_threshold 83.89029502868652 type value cosine_ap 76.28529631089978 type value cosine_f1 72.18316549496485 type value cosine_f1_threshold 79.37869429588318 type value cosine_precision 60.79372699631941 type value cosine_recall 88.82394201543138 type value dot_accuracy 69.29645219482863 type value dot_accuracy_threshold 83.890300989151 type value dot_ap 76.28533525182606 type value dot_f1 72.18316549496485 type value dot_f1_threshold 79.37869429588318 type value dot_precision 60.79372699631941 type value dot_recall 88.82394201543138 type value euclidean_accuracy 69.29645219482863 type value euclidean_accuracy_threshold 56.762146949768066 type value euclidean_ap 76.28547969937172 type value euclidean_f1 72.18316549496485 type value euclidean_f1_threshold 64.22040462493896 type value euclidean_precision 60.79372699631941 type value euclidean_recall 88.82394201543138 type value main_score 76.28547969937172 type value manhattan_accuracy 68.86349969933855 type value manhattan_accuracy_threshold 1325.539207458496 type value manhattan_ap 75.73527179489312 type value manhattan_f1 71.93284110448064 type value manhattan_f1_threshold 1450.2345085144043 type value manhattan_precision 63.386809269162214 type value manhattan_recall 83.14238952536824 type value max_ap 76.28547969937172 type value max_f1 72.18316549496485 type value max_precision 63.386809269162214 type value max_recall 88.82394201543138 type value similarity_accuracy 69.29645219482863 type value similarity_accuracy_threshold 83.89029502868652 type value similarity_ap 76.28529631089978 type value similarity_f1 72.18316549496485 type value similarity_f1_threshold 79.37869429588318 type value similarity_precision 60.79372699631941 type value similarity_recall 88.82394201543138 type PairClassification dataset metrics task config name revision split type default MTEB CovidRetrieval 1271c7809071a13532e05f25fb53511ffce77117 dev C-MTEB/CovidRetrieval type value main_score 82.76599999999999 type value map_at_1 70.99600000000001 type value map_at_10 79.022 type value map_at_100 79.262 type value map_at_1000 79.266 type value map_at_20 79.211 type value map_at_3 77.081 type value map_at_5 78.348 type value mrr_at_1 71.12750263435194 type value mrr_at_10 79.00563667686959 type value mrr_at_100 79.24545000482046 type value mrr_at_1000 79.24986213861123 type value mrr_at_20 79.19503716749968 type value mrr_at_3 77.1338250790306 type value mrr_at_5 78.38250790305591 type value nauc_map_at_1000_diff1 79.78007097062118 type value nauc_map_at_1000_max 31.495494389521216 type value nauc_map_at_1000_std -44.554113523471585 type value nauc_map_at_100_diff1 79.77901003479913 type value nauc_map_at_100_max 31.501728637681925 type value nauc_map_at_100_std -44.54526589087225 type value nauc_map_at_10_diff1 79.70465086616332 type value nauc_map_at_10_max 31.447942385382856 type value nauc_map_at_10_std -44.86102015819248 type value nauc_map_at_1_diff1 81.89774804895447 type value nauc_map_at_1_max 29.53109235427305 type value nauc_map_at_1_std -42.80277721451948 type value nauc_map_at_20_diff1 79.77871635635559 type value nauc_map_at_20_max 31.560274527206733 type value nauc_map_at_20_std -44.55008236120152 type value nauc_map_at_3_diff1 79.37871528079008 type value nauc_map_at_3_max 30.314627717947655 type value nauc_map_at_3_std -46.583081505018214 type value nauc_map_at_5_diff1 79.47410569600237 type value nauc_map_at_5_max 30.717452787943255 type value nauc_map_at_5_std -45.56487302807213 type value nauc_mrr_at_1000_diff1 79.83396133475738 type value nauc_mrr_at_1000_max 31.902081193300802 type value nauc_mrr_at_1000_std -44.32825329012893 type value nauc_mrr_at_100_diff1 79.832888351025 type value nauc_mrr_at_100_max 31.90821451879506 type value nauc_mrr_at_100_std -44.31946551133598 type value nauc_mrr_at_10_diff1 79.75766328526763 type value nauc_mrr_at_10_max 31.84709271229474 type value nauc_mrr_at_10_std -44.64251370779262 type value nauc_mrr_at_1_diff1 81.88675621341875 type value nauc_mrr_at_1_max 30.624768062722435 type value nauc_mrr_at_1_std -41.826968180693456 type value nauc_mrr_at_20_diff1 79.83221800317402 type value nauc_mrr_at_20_max 31.96340672339527 type value nauc_mrr_at_20_std -44.32956320098315 type value nauc_mrr_at_3_diff1 79.34629346809106 type value nauc_mrr_at_3_max 31.358295528236113 type value nauc_mrr_at_3_std -45.97803582281396 type value nauc_mrr_at_5_diff1 79.494177213373 type value nauc_mrr_at_5_max 31.52236804483443 type value nauc_mrr_at_5_std -45.138775893398694 type value nauc_ndcg_at_1000_diff1 79.42223230573576 type value nauc_ndcg_at_1000_max 32.28843903409106 type value nauc_ndcg_at_1000_std -44.3133954110294 type value nauc_ndcg_at_100_diff1 79.3929907054809 type value nauc_ndcg_at_100_max 32.49291426150998 type value nauc_ndcg_at_100_std -43.996604718501075 type value nauc_ndcg_at_10_diff1 79.11644773352661 type value nauc_ndcg_at_10_max 32.54744027915217 type value nauc_ndcg_at_10_std -45.44820798746672 type value nauc_ndcg_at_1_diff1 81.71471193659804 type value nauc_ndcg_at_1_max 30.56723762753589 type value nauc_ndcg_at_1_std -42.00582595178881 type value nauc_ndcg_at_20_diff1 79.34070754205227 type value nauc_ndcg_at_20_max 33.08175655505984 type value nauc_ndcg_at_20_std -43.93297429354463 type value nauc_ndcg_at_3_diff1 78.41040890432154 type value nauc_ndcg_at_3_max 30.540602587995053 type value nauc_ndcg_at_3_std -48.682741281966244 type value nauc_ndcg_at_5_diff1 78.52045059102817 type value nauc_ndcg_at_5_max 31.145620595701786 type value nauc_ndcg_at_5_std -46.96161213475506 type value nauc_precision_at_1000_diff1 -26.20700295711843 type value nauc_precision_at_1000_max 50.992072309587066 type value nauc_precision_at_1000_std 49.034232966809896 type value nauc_precision_at_100_diff1 -1.2318650992746658 type value nauc_precision_at_100_max 54.103623972545876 type value nauc_precision_at_100_std 38.158651434354105 type value nauc_precision_at_10_diff1 47.40081635911143 type value nauc_precision_at_10_max 46.01760789553407 type value nauc_precision_at_10_std -22.545587533051467 type value nauc_precision_at_1_diff1 81.71471193659804 type value nauc_precision_at_1_max 30.56723762753589 type value nauc_precision_at_1_std -42.00582595178881 type value nauc_precision_at_20_diff1 31.902645462266044 type value nauc_precision_at_20_max 60.06037928799191 type value nauc_precision_at_20_std 10.125381568485691 type value nauc_precision_at_3_diff1 70.23181696295782 type value nauc_precision_at_3_max 31.33307476962615 type value nauc_precision_at_3_std -52.773523783308995 type value nauc_precision_at_5_diff1 63.24118340779976 type value nauc_precision_at_5_max 35.536460706118284 type value nauc_precision_at_5_std -43.859100503715496 type value nauc_recall_at_1000_diff1 63.10783066308766 type value nauc_recall_at_1000_max 64.17746555050037 type value nauc_recall_at_1000_std -1.1314627694685895 type value nauc_recall_at_100_diff1 70.70747402244945 type value nauc_recall_at_100_max 63.81462634298472 type value nauc_recall_at_100_std 2.7329437124855858 type value nauc_recall_at_10_diff1 74.5724683430861 type value nauc_recall_at_10_max 42.06028697147503 type value nauc_recall_at_10_std -50.426163431789384 type value nauc_recall_at_1_diff1 81.89774804895447 type value nauc_recall_at_1_max 29.53109235427305 type value nauc_recall_at_1_std -42.80277721451948 type value nauc_recall_at_20_diff1 74.1386367152198 type value nauc_recall_at_20_max 60.26605112943992 type value nauc_recall_at_20_std -24.167905489617926 type value nauc_recall_at_3_diff1 74.68360442418249 type value nauc_recall_at_3_max 29.73174978017023 type value nauc_recall_at_3_std -58.048521143234844 type value nauc_recall_at_5_diff1 73.33434605574439 type value nauc_recall_at_5_max 31.829043506426963 type value nauc_recall_at_5_std -55.33176739081927 type value ndcg_at_1 71.233 type value ndcg_at_10 82.76599999999999 type value ndcg_at_100 83.799 type value ndcg_at_1000 83.898 type value ndcg_at_20 83.44 type value ndcg_at_3 79.03999999999999 type value ndcg_at_5 81.285 type value precision_at_1 71.233 type value precision_at_10 9.526 type value precision_at_100 0.9990000000000001 type value precision_at_1000 0.101 type value precision_at_20 4.8950000000000005 type value precision_at_3 28.346 type value precision_at_5 18.124000000000002 type value recall_at_1 70.99600000000001 type value recall_at_10 94.31 type value recall_at_100 98.84100000000001 type value recall_at_1000 99.579 type value recall_at_20 96.944 type value recall_at_3 84.589 type value recall_at_5 89.98899999999999 type Retrieval dataset metrics task config name revision split type default MTEB DuRetrieval a1a333e290fe30b10f3f56498e3a0d911a693ced dev C-MTEB/DuRetrieval type value main_score 82.353 type value map_at_1 23.408 type value map_at_10 73.302 type value map_at_100 76.532 type value map_at_1000 76.578 type value map_at_20 75.765 type value map_at_3 49.297999999999995 type value map_at_5 62.96000000000001 type value mrr_at_1 83.3 type value mrr_at_10 88.85841269841264 type value mrr_at_100 88.937851229216 type value mrr_at_1000 88.94253811030754 type value mrr_at_20 88.90522789194803 type value mrr_at_3 88.31666666666662 type value mrr_at_5 88.66416666666662 type value nauc_map_at_1000_diff1 3.978108165433077 type value nauc_map_at_1000_max 32.84013060265069 type value nauc_map_at_1000_std 17.104374545928255 type value nauc_map_at_100_diff1 3.9594456007844183 type value nauc_map_at_100_max 32.84323698444807 type value nauc_map_at_100_std 17.083360165851175 type value nauc_map_at_10_diff1 6.564428602685249 type value nauc_map_at_10_max 29.490007273766956 type value nauc_map_at_10_std 6.955854455105477 type value nauc_map_at_1_diff1 43.01902060700144 type value nauc_map_at_1_max -8.940094269879843 type value nauc_map_at_1_std -28.063233795166276 type value nauc_map_at_20_diff1 4.446904145850981 type value nauc_map_at_20_max 32.47424290913474 type value nauc_map_at_20_std 14.957146942696257 type value nauc_map_at_3_diff1 25.91745605988797 type value nauc_map_at_3_max 3.661124903759869 type value nauc_map_at_3_std -21.936610233451646 type value nauc_map_at_5_diff1 16.629939273347865 type value nauc_map_at_5_max 14.666913498454564 type value nauc_map_at_5_std -12.39941441022446 type value nauc_mrr_at_1000_diff1 26.08525262735903 type value nauc_mrr_at_1000_max 47.86393129438558 type value nauc_mrr_at_1000_std 28.811634091001743 type value nauc_mrr_at_100_diff1 26.081836904532153 type value nauc_mrr_at_100_max 47.880134050815 type value nauc_mrr_at_100_std 28.828980969011475 type value nauc_mrr_at_10_diff1 26.09549377249783 type value nauc_mrr_at_10_max 48.11004429436051 type value nauc_mrr_at_10_std 29.041772733561455 type value nauc_mrr_at_1_diff1 26.095576390896717 type value nauc_mrr_at_1_max 40.102786808829485 type value nauc_mrr_at_1_std 21.16142603421125 type value nauc_mrr_at_20_diff1 26.078553311053394 type value nauc_mrr_at_20_max 47.9955055491724 type value nauc_mrr_at_20_std 28.92844826033336 type value nauc_mrr_at_3_diff1 25.736821420614447 type value nauc_mrr_at_3_max 48.30695057366758 type value nauc_mrr_at_3_std 29.295726311215475 type value nauc_mrr_at_5_diff1 25.979034861669714 type value nauc_mrr_at_5_max 48.500915285456344 type value nauc_mrr_at_5_std 29.449704923164106 type value nauc_ndcg_at_1000_diff1 6.624272455812551 type value nauc_ndcg_at_1000_max 41.526519286613414 type value nauc_ndcg_at_1000_std 27.91983541845217 type value nauc_ndcg_at_100_diff1 6.033169661320914 type value nauc_ndcg_at_100_max 41.6841728152419 type value nauc_ndcg_at_100_std 28.35967524719135 type value nauc_ndcg_at_10_diff1 5.627968389448389 type value nauc_ndcg_at_10_max 37.18261001317417 type value nauc_ndcg_at_10_std 19.757054878692408 type value nauc_ndcg_at_1_diff1 26.095576390896717 type value nauc_ndcg_at_1_max 40.102786808829485 type value nauc_ndcg_at_1_std 21.16142603421125 type value nauc_ndcg_at_20_diff1 5.678380464964442 type value nauc_ndcg_at_20_max 40.70268508824627 type value nauc_ndcg_at_20_std 25.003203457508622 type value nauc_ndcg_at_3_diff1 5.7196343030730645 type value nauc_ndcg_at_3_max 34.50950904905902 type value nauc_ndcg_at_3_std 20.099411226966403 type value nauc_ndcg_at_5_diff1 7.398974214665505 type value nauc_ndcg_at_5_max 31.777872881596885 type value nauc_ndcg_at_5_std 14.212532410116573 type value nauc_precision_at_1000_diff1 -26.784369186388286 type value nauc_precision_at_1000_max 20.9055343942668 type value nauc_precision_at_1000_std 48.97851074406537 type value nauc_precision_at_100_diff1 -27.79381730090699 type value nauc_precision_at_100_max 22.80005440633608 type value nauc_precision_at_100_std 50.935594672026795 type value nauc_precision_at_10_diff1 -30.285772529280557 type value nauc_precision_at_10_max 32.73392928068347 type value nauc_precision_at_10_std 47.96878369413408 type value nauc_precision_at_1_diff1 26.095576390896717 type value nauc_precision_at_1_max 40.102786808829485 type value nauc_precision_at_1_std 21.16142603421125 type value nauc_precision_at_20_diff1 -28.93118180068221 type value nauc_precision_at_20_max 27.34554979821627 type value nauc_precision_at_20_std 50.768062841591245 type value nauc_precision_at_3_diff1 -20.842604987632818 type value nauc_precision_at_3_max 38.567385349160865 type value nauc_precision_at_3_std 34.962189381111585 type value nauc_precision_at_5_diff1 -27.39434681486595 type value nauc_precision_at_5_max 36.46059763518038 type value nauc_precision_at_5_std 39.893251684847286 type value nauc_recall_at_1000_diff1 -11.949093496228018 type value nauc_recall_at_1000_max 73.88534051191724 type value nauc_recall_at_1000_std 74.63173870654316 type value nauc_recall_at_100_diff1 -10.612653444299633 type value nauc_recall_at_100_max 55.332461824335255 type value nauc_recall_at_100_std 55.6971441098854 type value nauc_recall_at_10_diff1 1.6381390695279527 type value nauc_recall_at_10_max 30.7773121587242 type value nauc_recall_at_10_std 5.983376763709044 type value nauc_recall_at_1_diff1 43.01902060700144 type value nauc_recall_at_1_max -8.940094269879843 type value nauc_recall_at_1_std -28.063233795166276 type value nauc_recall_at_20_diff1 -3.5879888483690268 type value nauc_recall_at_20_max 42.56780359254684 type value nauc_recall_at_20_std 28.64620011473346 type value nauc_recall_at_3_diff1 24.423753178927363 type value nauc_recall_at_3_max 0.28631207577281326 type value nauc_recall_at_3_std -24.79099042560129 type value nauc_recall_at_5_diff1 15.716357450134492 type value nauc_recall_at_5_max 9.923967009889193 type value nauc_recall_at_5_std -18.11714448988651 type value ndcg_at_1 83.3 type value ndcg_at_10 82.353 type value ndcg_at_100 85.952 type value ndcg_at_1000 86.393 type value ndcg_at_20 84.333 type value ndcg_at_3 79.128 type value ndcg_at_5 78.96300000000001 type value precision_at_1 83.3 type value precision_at_10 40.36 type value precision_at_100 4.769 type value precision_at_1000 0.488 type value precision_at_20 22.295 type value precision_at_3 71.25 type value precision_at_5 61.18 type value recall_at_1 23.408 type value recall_at_10 85.44800000000001 type value recall_at_100 96.712 type value recall_at_1000 98.988 type value recall_at_20 91.304 type value recall_at_3 52.65 type value recall_at_5 69.81 type Retrieval dataset metrics task config name revision split type default MTEB EcomRetrieval 687de13dc7294d6fd9be10c6945f9e8fec8166b9 dev C-MTEB/EcomRetrieval type value main_score 63.712999999999994 type value map_at_1 49.0 type value map_at_10 58.620000000000005 type value map_at_100 59.183 type value map_at_1000 59.19799999999999 type value map_at_20 58.948 type value map_at_3 55.883 type value map_at_5 57.452999999999996 type value mrr_at_1 49.0 type value mrr_at_10 58.61988095238089 type value mrr_at_100 59.18251462760907 type value mrr_at_1000 59.1981896580556 type value mrr_at_20 58.94805232134562 type value mrr_at_3 55.883333333333304 type value mrr_at_5 57.4533333333333 type value nauc_map_at_1000_diff1 60.33101842801658 type value nauc_map_at_1000_max 19.502683068762945 type value nauc_map_at_1000_std -9.052741690420172 type value nauc_map_at_100_diff1 60.320202163437884 type value nauc_map_at_100_max 19.511425958183473 type value nauc_map_at_100_std -9.046775711361885 type value nauc_map_at_10_diff1 60.32228179956949 type value nauc_map_at_10_max 19.6159978656515 type value nauc_map_at_10_std -9.132522477977544 type value nauc_map_at_1_diff1 61.89621977613427 type value nauc_map_at_1_max 15.015734335373715 type value nauc_map_at_1_std -12.641774992365185 type value nauc_map_at_20_diff1 60.351130642660486 type value nauc_map_at_20_max 19.433343357030232 type value nauc_map_at_20_std -9.21598413872683 type value nauc_map_at_3_diff1 60.26725821298107 type value nauc_map_at_3_max 18.3498595109406 type value nauc_map_at_3_std -10.051517839346984 type value nauc_map_at_5_diff1 60.164921439673925 type value nauc_map_at_5_max 18.593900545400267 type value nauc_map_at_5_std -9.934110598947624 type value nauc_mrr_at_1000_diff1 60.33101842801658 type value nauc_mrr_at_1000_max 19.502683068762945 type value nauc_mrr_at_1000_std -9.052741690420172 type value nauc_mrr_at_100_diff1 60.320202163437884 type value nauc_mrr_at_100_max 19.511425958183473 type value nauc_mrr_at_100_std -9.046775711361885 type value nauc_mrr_at_10_diff1 60.32228179956949 type value nauc_mrr_at_10_max 19.6159978656515 type value nauc_mrr_at_10_std -9.132522477977544 type value nauc_mrr_at_1_diff1 61.89621977613427 type value nauc_mrr_at_1_max 15.015734335373715 type value nauc_mrr_at_1_std -12.641774992365185 type value nauc_mrr_at_20_diff1 60.351130642660486 type value nauc_mrr_at_20_max 19.433343357030232 type value nauc_mrr_at_20_std -9.21598413872683 type value nauc_mrr_at_3_diff1 60.26725821298107 type value nauc_mrr_at_3_max 18.3498595109406 type value nauc_mrr_at_3_std -10.051517839346984 type value nauc_mrr_at_5_diff1 60.164921439673925 type value nauc_mrr_at_5_max 18.593900545400267 type value nauc_mrr_at_5_std -9.934110598947624 type value nauc_ndcg_at_1000_diff1 60.190733838614676 type value nauc_ndcg_at_1000_max 22.361539210340222 type value nauc_ndcg_at_1000_std -5.745163462434749 type value nauc_ndcg_at_100_diff1 59.89473232352801 type value nauc_ndcg_at_100_max 22.68282893350434 type value nauc_ndcg_at_100_std -5.4179387740783 type value nauc_ndcg_at_10_diff1 60.07971889322107 type value nauc_ndcg_at_10_max 22.591286648072977 type value nauc_ndcg_at_10_std -6.68500894448089 type value nauc_ndcg_at_1_diff1 61.89621977613427 type value nauc_ndcg_at_1_max 15.015734335373715 type value nauc_ndcg_at_1_std -12.641774992365185 type value nauc_ndcg_at_20_diff1 60.182873920240475 type value nauc_ndcg_at_20_max 21.964898434175247 type value nauc_ndcg_at_20_std -6.906365610289816 type value nauc_ndcg_at_3_diff1 59.8208566369894 type value nauc_ndcg_at_3_max 19.388884168625417 type value nauc_ndcg_at_3_std -9.151250601081255 type value nauc_ndcg_at_5_diff1 59.599342583351955 type value nauc_ndcg_at_5_max 19.8910854628725 type value nauc_ndcg_at_5_std -8.885354650481215 type value nauc_precision_at_1000_diff1 63.58164887576627 type value nauc_precision_at_1000_max 92.23383046912454 type value nauc_precision_at_1000_std 87.13881949176067 type value nauc_precision_at_100_diff1 53.73002142033278 type value nauc_precision_at_100_max 70.37128576920941 type value nauc_precision_at_100_std 55.41687263140533 type value nauc_precision_at_10_diff1 59.41629120257138 type value nauc_precision_at_10_max 38.24957021696883 type value nauc_precision_at_10_std 6.335412380239172 type value nauc_precision_at_1_diff1 61.89621977613427 type value nauc_precision_at_1_max 15.015734335373715 type value nauc_precision_at_1_std -12.641774992365185 type value nauc_precision_at_20_diff1 59.95367722749617 type value nauc_precision_at_20_max 38.11970211089507 type value nauc_precision_at_20_std 8.468361991180146 type value nauc_precision_at_3_diff1 58.418401476502524 type value nauc_precision_at_3_max 22.708479411978058 type value nauc_precision_at_3_std -6.238867799833925 type value nauc_precision_at_5_diff1 57.54249152786323 type value nauc_precision_at_5_max 24.64947877432984 type value nauc_precision_at_5_std -5.018047100033905 type value nauc_recall_at_1000_diff1 63.581648875766604 type value nauc_recall_at_1000_max 92.23383046912458 type value nauc_recall_at_1000_std 87.13881949176098 type value nauc_recall_at_100_diff1 53.73002142033278 type value nauc_recall_at_100_max 70.37128576920976 type value nauc_recall_at_100_std 55.41687263140555 type value nauc_recall_at_10_diff1 59.41629120257145 type value nauc_recall_at_10_max 38.2495702169689 type value nauc_recall_at_10_std 6.335412380239176 type value nauc_recall_at_1_diff1 61.89621977613427 type value nauc_recall_at_1_max 15.015734335373715 type value nauc_recall_at_1_std -12.641774992365185 type value nauc_recall_at_20_diff1 59.95367722749639 type value nauc_recall_at_20_max 38.11970211089514 type value nauc_recall_at_20_std 8.468361991180268 type value nauc_recall_at_3_diff1 58.41840147650248 type value nauc_recall_at_3_max 22.708479411978043 type value nauc_recall_at_3_std -6.238867799833981 type value nauc_recall_at_5_diff1 57.542491527863206 type value nauc_recall_at_5_max 24.649478774330014 type value nauc_recall_at_5_std -5.018047100033782 type value ndcg_at_1 49.0 type value ndcg_at_10 63.712999999999994 type value ndcg_at_100 66.523 type value ndcg_at_1000 66.922 type value ndcg_at_20 64.904 type value ndcg_at_3 58.099000000000004 type value ndcg_at_5 60.913 type value precision_at_1 49.0 type value precision_at_10 7.99 type value precision_at_100 0.932 type value precision_at_1000 0.096 type value precision_at_20 4.2299999999999995 type value precision_at_3 21.5 type value precision_at_5 14.26 type value recall_at_1 49.0 type value recall_at_10 79.9 type value recall_at_100 93.2 type value recall_at_1000 96.3 type value recall_at_20 84.6 type value recall_at_3 64.5 type value recall_at_5 71.3 type Retrieval dataset metrics task config name revision split type default MTEB IFlyTek 421605374b29664c5fc098418fe20ada9bd55f8a validation C-MTEB/IFlyTek-classification type value accuracy 49.188149288187766 type value f1 35.82742058478872 type value f1_weighted 46.33812923348324 type value main_score 49.188149288187766 type Classification dataset metrics task config name revision split type default MTEB JDReview b7c64bd89eb87f8ded463478346f76731f07bf8b test C-MTEB/JDReview-classification type value accuracy 83.45215759849907 type value ap 49.602287249765666 type value ap_weighted 49.602287249765666 type value f1 77.84519218126933 type value f1_weighted 84.83784419250833 type value main_score 83.45215759849907 type Classification dataset metrics task config name revision split type default MTEB LCQMC 17f9b096f80380fce5ed12a9be8be7784b337daf test C-MTEB/LCQMC type value cosine_pearson 66.78399631818323 type value cosine_spearman 70.38648345929874 type value euclidean_pearson 68.79036522204457 type value euclidean_spearman 70.38649454085622 type value main_score 70.38648345929874 type value manhattan_pearson 68.74927335399974 type value manhattan_spearman 70.3453886791424 type value pearson 66.78399631818323 type value spearman 70.38648345929874 type STS dataset metrics task config name revision split type default MTEB MMarcoReranking 8e0c766dbe9e16e1d221116a3f36795fbade07f6 dev C-MTEB/Mmarco-reranking type value main_score 26.991570930656568 type value map 26.991570930656568 type value mrr 25.460714285714285 type value nAUC_map_diff1 12.174277381054415 type value nAUC_map_max 5.768145859960792 type value nAUC_map_std -0.6863999286086584 type value nAUC_mrr_diff1 11.83053464449912 type value nAUC_mrr_max 4.893060023643725 type value nAUC_mrr_std -0.22755376963555723 type Reranking dataset metrics task config name revision split type default MTEB MMarcoRetrieval 539bbde593d947e2a124ba72651aafc09eb33fc2 dev C-MTEB/MMarcoRetrieval type value main_score 78.679 type value map_at_1 65.349 type value map_at_10 74.802 type value map_at_100 75.141 type value map_at_1000 75.151 type value map_at_20 75.03999999999999 type value map_at_3 72.831 type value map_at_5 74.09400000000001 type value mrr_at_1 67.55014326647564 type value mrr_at_10 75.31912038932084 type value mrr_at_100 75.6225574951573 type value mrr_at_1000 75.63176308010398 type value mrr_at_20 75.53574557856176 type value mrr_at_3 73.59598853868198 type value mrr_at_5 74.70343839541526 type value nauc_map_at_1000_diff1 77.81972509758704 type value nauc_map_at_1000_max 27.445457824343595 type value nauc_map_at_1000_std -18.60670002314929 type value nauc_map_at_100_diff1 77.81776087022583 type value nauc_map_at_100_max 27.465677796741794 type value nauc_map_at_100_std -18.574455053179566 type value nauc_map_at_10_diff1 77.668921503636 type value nauc_map_at_10_max 27.564476726876563 type value nauc_map_at_10_std -18.67577233314456 type value nauc_map_at_1_diff1 80.13251752826227 type value nauc_map_at_1_max 19.700940114548352 type value nauc_map_at_1_std -24.276498497801104 type value nauc_map_at_20_diff1 77.76444686257037 type value nauc_map_at_20_max 27.507355610895434 type value nauc_map_at_20_std -18.570029885207234 type value nauc_map_at_3_diff1 77.62870706241021 type value nauc_map_at_3_max 25.979199504514654 type value nauc_map_at_3_std -20.480776195240768 type value nauc_map_at_5_diff1 77.68046637184071 type value nauc_map_at_5_max 27.068345296401887 type value nauc_map_at_5_std -19.515458511154968 type value nauc_mrr_at_1000_diff1 78.12673001253819 type value nauc_mrr_at_1000_max 28.23584877768183 type value nauc_mrr_at_1000_std -17.765605843184606 type value nauc_mrr_at_100_diff1 78.12476632443614 type value nauc_mrr_at_100_max 28.255499574563654 type value nauc_mrr_at_100_std -17.73302695902061 type value nauc_mrr_at_10_diff1 77.98552897771079 type value nauc_mrr_at_10_max 28.433270245298903 type value nauc_mrr_at_10_std -17.721467674164725 type value nauc_mrr_at_1_diff1 80.74164178463916 type value nauc_mrr_at_1_max 23.400992011183135 type value nauc_mrr_at_1_std -23.155846305708668 type value nauc_mrr_at_20_diff1 78.08519488707572 type value nauc_mrr_at_20_max 28.305974768972476 type value nauc_mrr_at_20_std -17.70766096956611 type value nauc_mrr_at_3_diff1 77.99203426607973 type value nauc_mrr_at_3_max 27.39053740753677 type value nauc_mrr_at_3_std -19.110899565832597 type value nauc_mrr_at_5_diff1 77.99012861357085 type value nauc_mrr_at_5_max 28.018453732422905 type value nauc_mrr_at_5_std -18.45275089190139 type value nauc_ndcg_at_1000_diff1 77.37899152370498 type value nauc_ndcg_at_1000_max 29.715512454119402 type value nauc_ndcg_at_1000_std -15.311768186844196 type value nauc_ndcg_at_100_diff1 77.30487512550962 type value nauc_ndcg_at_100_max 30.358291073116767 type value nauc_ndcg_at_100_std -14.276238712942787 type value nauc_ndcg_at_10_diff1 76.55306779956729 type value nauc_ndcg_at_10_max 31.003218536597576 type value nauc_ndcg_at_10_std -14.528637377688142 type value nauc_ndcg_at_1_diff1 80.74164178463916 type value nauc_ndcg_at_1_max 23.400992011183135 type value nauc_ndcg_at_1_std -23.155846305708668 type value nauc_ndcg_at_20_diff1 76.92359358217516 type value nauc_ndcg_at_20_max 30.734983558658648 type value nauc_ndcg_at_20_std -14.12117266760052 type value nauc_ndcg_at_3_diff1 76.65174056138369 type value nauc_ndcg_at_3_max 27.744998584618365 type value nauc_ndcg_at_3_std -18.596857381234265 type value nauc_ndcg_at_5_diff1 76.64434516875298 type value nauc_ndcg_at_5_max 29.580949778455096 type value nauc_ndcg_at_5_std -16.820146947848347 type value nauc_precision_at_1000_diff1 -15.819998326963425 type value nauc_precision_at_1000_max 22.790060032171432 type value nauc_precision_at_1000_std 25.646210332652032 type value nauc_precision_at_100_diff1 -3.225658983047692 type value nauc_precision_at_100_max 31.046785086458396 type value nauc_precision_at_100_std 30.64496213174489 type value nauc_precision_at_10_diff1 22.399826113454544 type value nauc_precision_at_10_max 37.17215584865757 type value nauc_precision_at_10_std 16.375879066453813 type value nauc_precision_at_1_diff1 80.74164178463916 type value nauc_precision_at_1_max 23.400992011183135 type value nauc_precision_at_1_std -23.155846305708668 type value nauc_precision_at_20_diff1 11.824890141102545 type value nauc_precision_at_20_max 35.7858012680296 type value nauc_precision_at_20_std 24.36537306318588 type value nauc_precision_at_3_diff1 46.964579254137156 type value nauc_precision_at_3_max 31.240508812172248 type value nauc_precision_at_3_std -4.790609954536406 type value nauc_precision_at_5_diff1 35.92331054363029 type value nauc_precision_at_5_max 34.58921599366064 type value nauc_precision_at_5_std 3.955705927038542 type value nauc_recall_at_1000_diff1 69.82124326053469 type value nauc_recall_at_1000_max 77.26332872982017 type value nauc_recall_at_1000_std 74.20589405678723 type value nauc_recall_at_100_diff1 71.09335151657598 type value nauc_recall_at_100_max 74.66551138520433 type value nauc_recall_at_100_std 62.296014312578606 type value nauc_recall_at_10_diff1 68.34266216578438 type value nauc_recall_at_10_max 51.776074855673635 type value nauc_recall_at_10_std 11.551590635685633 type value nauc_recall_at_1_diff1 80.13251752826227 type value nauc_recall_at_1_max 19.700940114548352 type value nauc_recall_at_1_std -24.276498497801104 type value nauc_recall_at_20_diff1 68.44098404116468 type value nauc_recall_at_20_max 58.0709257934264 type value nauc_recall_at_20_std 27.20288447881239 type value nauc_recall_at_3_diff1 72.224364274587 type value nauc_recall_at_3_max 32.11973511168104 type value nauc_recall_at_3_std -13.287781131985849 type value nauc_recall_at_5_diff1 70.97684486885963 type value nauc_recall_at_5_max 39.47238239221433 type value nauc_recall_at_5_std -5.749985209368605 type value ndcg_at_1 67.55 type value ndcg_at_10 78.679 type value ndcg_at_100 80.16 type value ndcg_at_1000 80.42 type value ndcg_at_20 79.50500000000001 type value ndcg_at_3 74.96199999999999 type value ndcg_at_5 77.093 type value precision_at_1 67.55 type value precision_at_10 9.589 type value precision_at_100 1.031 type value precision_at_1000 0.105 type value precision_at_20 4.966 type value precision_at_3 28.319 type value precision_at_5 18.129 type value recall_at_1 65.349 type value recall_at_10 90.10000000000001 type value recall_at_100 96.685 type value recall_at_1000 98.714 type value recall_at_20 93.298 type value recall_at_3 80.324 type value recall_at_5 85.37700000000001 type Retrieval dataset metrics task config name revision split type zh-CN MTEB MassiveIntentClassification (zh-CN) 4672e20407010da34463acc759c162ca9734bca6 test mteb/amazon_massive_intent type value accuracy 70.94149293880295 type value f1 67.43015916458866 type value f1_weighted 70.02165762549619 type value main_score 70.94149293880295 type Classification dataset metrics task config name revision split type zh-TW MTEB MassiveIntentClassification (zh-TW) 4672e20407010da34463acc759c162ca9734bca6 test mteb/amazon_massive_intent type value accuracy 63.05312710154675 type value f1 61.11778922874984 type value f1_weighted 61.425454449692396 type value main_score 63.05312710154675 type Classification dataset metrics task config name revision split type zh-CN MTEB MassiveScenarioClassification (zh-CN) fad2c6e8459f9e1c45d9315f4953d921437d70f8 test mteb/amazon_massive_scenario type value accuracy 77.79757901815736 type value f1 76.85610655879204 type value f1_weighted 77.36623686607157 type value main_score 77.79757901815736 type Classification dataset metrics task config name revision split type zh-TW MTEB MassiveScenarioClassification (zh-TW) fad2c6e8459f9e1c45d9315f4953d921437d70f8 test mteb/amazon_massive_scenario type value accuracy 71.34498991257566 type value f1 71.42538497861686 type value f1_weighted 70.47776598531958 type value main_score 71.34498991257566 type Classification dataset metrics task config name revision split type default MTEB MedicalRetrieval 2039188fb5800a9803ba5048df7b76e6fb151fc6 dev C-MTEB/MedicalRetrieval type value main_score 57.528999999999996 type value map_at_1 48.699999999999996 type value map_at_10 54.674 type value map_at_100 55.187 type value map_at_1000 55.24 type value map_at_20 54.933 type value map_at_3 53.367 type value map_at_5 54.081999999999994 type value mrr_at_1 48.8 type value mrr_at_10 54.71369047619046 type value mrr_at_100 55.23606881716415 type value mrr_at_1000 55.2887596380029 type value mrr_at_20 54.98226974307081 type value mrr_at_3 53.41666666666666 type value mrr_at_5 54.131666666666646 type value nauc_map_at_1000_diff1 79.392997677128 type value nauc_map_at_1000_max 47.4042544614244 type value nauc_map_at_1000_std 23.2164546714886 type value nauc_map_at_100_diff1 79.3811285055918 type value nauc_map_at_100_max 47.399489637525214 type value nauc_map_at_100_std 23.24298678047571 type value nauc_map_at_10_diff1 79.51795702164893 type value nauc_map_at_10_max 47.3775323018549 type value nauc_map_at_10_std 22.863584607876017 type value nauc_map_at_1_diff1 82.77387889149895 type value nauc_map_at_1_max 48.92316018033766 type value nauc_map_at_1_std 20.670920881420933 type value nauc_map_at_20_diff1 79.36321354500926 type value nauc_map_at_20_max 47.347135287818695 type value nauc_map_at_20_std 23.128792587733724 type value nauc_map_at_3_diff1 79.89693675044646 type value nauc_map_at_3_max 47.999519454025815 type value nauc_map_at_3_std 22.67285215587248 type value nauc_map_at_5_diff1 79.72880868956226 type value nauc_map_at_5_max 47.870829359727615 type value nauc_map_at_5_std 22.75976001331719 type value nauc_mrr_at_1000_diff1 79.2558524289943 type value nauc_mrr_at_1000_max 47.68193948210489 type value nauc_mrr_at_1000_std 23.488171939833503 type value nauc_mrr_at_100_diff1 79.2441760972466 type value nauc_mrr_at_100_max 47.67677923765432 type value nauc_mrr_at_100_std 23.51432250784555 type value nauc_mrr_at_10_diff1 79.39423493974832 type value nauc_mrr_at_10_max 47.672297066929545 type value nauc_mrr_at_10_std 23.13845505800058 type value nauc_mrr_at_1_diff1 82.51854957699533 type value nauc_mrr_at_1_max 49.43475537911197 type value nauc_mrr_at_1_std 21.172657021240443 type value nauc_mrr_at_20_diff1 79.22702612117199 type value nauc_mrr_at_20_max 47.62286080846738 type value nauc_mrr_at_20_std 23.398587017649174 type value nauc_mrr_at_3_diff1 79.76301529177348 type value nauc_mrr_at_3_max 48.26663425470944 type value nauc_mrr_at_3_std 22.935349467987145 type value nauc_mrr_at_5_diff1 79.5934610019844 type value nauc_mrr_at_5_max 48.1407033814883 type value nauc_mrr_at_5_std 23.025008156084695 type value nauc_ndcg_at_1000_diff1 77.97548063568358 type value nauc_ndcg_at_1000_max 46.670156188276266 type value nauc_ndcg_at_1000_std 25.32524568996684 type value nauc_ndcg_at_100_diff1 77.58788261282791 type value nauc_ndcg_at_100_max 46.366231150510664 type value nauc_ndcg_at_100_std 26.02842093987038 type value nauc_ndcg_at_10_diff1 78.15883898742274 type value nauc_ndcg_at_10_max 46.181496192291974 type value nauc_ndcg_at_10_std 23.997358704992077 type value nauc_ndcg_at_1_diff1 82.77387889149895 type value nauc_ndcg_at_1_max 48.92316018033766 type value nauc_ndcg_at_1_std 20.670920881420933 type value nauc_ndcg_at_20_diff1 77.51209948232727 type value nauc_ndcg_at_20_max 46.02903895633775 type value nauc_ndcg_at_20_std 25.023178998194467 type value nauc_ndcg_at_3_diff1 79.0464751622174 type value nauc_ndcg_at_3_max 47.65456262552185 type value nauc_ndcg_at_3_std 23.50005981191216 type value nauc_ndcg_at_5_diff1 78.73621060890696 type value nauc_ndcg_at_5_max 47.4490746627881 type value nauc_ndcg_at_5_std 23.70727530773819 type value nauc_precision_at_1000_diff1 63.42066238259988 type value nauc_precision_at_1000_max 43.54369198659821 type value nauc_precision_at_1000_std 55.676388202339524 type value nauc_precision_at_100_diff1 67.14856074074835 type value nauc_precision_at_100_max 40.92023184354666 type value nauc_precision_at_100_std 45.790641988757145 type value nauc_precision_at_10_diff1 73.22243545156664 type value nauc_precision_at_10_max 41.458823923773686 type value nauc_precision_at_10_std 28.142697919198138 type value nauc_precision_at_1_diff1 82.77387889149895 type value nauc_precision_at_1_max 48.92316018033766 type value nauc_precision_at_1_std 20.670920881420933 type value nauc_precision_at_20_diff1 69.5822714276579 type value nauc_precision_at_20_max 40.258145844180724 type value nauc_precision_at_20_std 33.443132096498665 type value nauc_precision_at_3_diff1 76.48729951428531 type value nauc_precision_at_3_max 46.58972515297812 type value nauc_precision_at_3_std 26.07700999310317 type value nauc_precision_at_5_diff1 75.58859746051998 type value nauc_precision_at_5_max 46.09484444567729 type value nauc_precision_at_5_std 26.82420134602608 type value nauc_recall_at_1000_diff1 63.42066238260002 type value nauc_recall_at_1000_max 43.543691986598645 type value nauc_recall_at_1000_std 55.67638820233998 type value nauc_recall_at_100_diff1 67.14856074074834 type value nauc_recall_at_100_max 40.92023184354673 type value nauc_recall_at_100_std 45.79064198875728 type value nauc_recall_at_10_diff1 73.22243545156665 type value nauc_recall_at_10_max 41.45882392377375 type value nauc_recall_at_10_std 28.14269791919819 type value nauc_recall_at_1_diff1 82.77387889149895 type value nauc_recall_at_1_max 48.92316018033766 type value nauc_recall_at_1_std 20.670920881420933 type value nauc_recall_at_20_diff1 69.58227142765797 type value nauc_recall_at_20_max 40.25814584418081 type value nauc_recall_at_20_std 33.443132096498665 type value nauc_recall_at_3_diff1 76.4872995142853 type value nauc_recall_at_3_max 46.589725152978076 type value nauc_recall_at_3_std 26.07700999310312 type value nauc_recall_at_5_diff1 75.58859746051999 type value nauc_recall_at_5_max 46.09484444567737 type value nauc_recall_at_5_std 26.8242013460261 type value ndcg_at_1 48.699999999999996 type value ndcg_at_10 57.528999999999996 type value ndcg_at_100 60.38 type value ndcg_at_1000 61.937 type value ndcg_at_20 58.518 type value ndcg_at_3 54.818999999999996 type value ndcg_at_5 56.101 type value precision_at_1 48.699999999999996 type value precision_at_10 6.65 type value precision_at_100 0.8059999999999999 type value precision_at_1000 0.093 type value precision_at_20 3.5249999999999995 type value precision_at_3 19.667 type value precision_at_5 12.42 type value recall_at_1 48.699999999999996 type value recall_at_10 66.5 type value recall_at_100 80.60000000000001 type value recall_at_1000 93.2 type value recall_at_20 70.5 type value recall_at_3 59.0 type value recall_at_5 62.1 type Retrieval dataset metrics task config name revision split type default MTEB MultilingualSentiment 46958b007a63fdbf239b7672c25d0bea67b5ea1a test C-MTEB/MultilingualSentiment-classification type value accuracy 78.08 type value f1 77.44308848942492 type value f1_weighted 77.44308848942492 type value main_score 78.08 type Classification dataset metrics task config name revision split type default MTEB Ocnli 66e76a618a34d6d565d5538088562851e6daa7ec validation C-MTEB/OCNLI type value cosine_accuracy 66.8651867893882 type value cosine_accuracy_threshold 84.34688448905945 type value cosine_ap 69.83287846115917 type value cosine_f1 71.33520074696546 type value cosine_f1_threshold 83.85992050170898 type value cosine_precision 63.93305439330545 type value cosine_recall 80.67581837381204 type value dot_accuracy 66.8651867893882 type value dot_accuracy_threshold 84.34690237045288 type value dot_ap 69.83287846115917 type value dot_f1 71.33520074696546 type value dot_f1_threshold 83.85992050170898 type value dot_precision 63.93305439330545 type value dot_recall 80.67581837381204 type value euclidean_accuracy 66.8651867893882 type value euclidean_accuracy_threshold 55.95196485519409 type value euclidean_ap 69.83287846115917 type value euclidean_f1 71.33520074696546 type value euclidean_f1_threshold 56.81561827659607 type value euclidean_precision 63.93305439330545 type value euclidean_recall 80.67581837381204 type value main_score 69.83287846115917 type value manhattan_accuracy 66.0530590146183 type value manhattan_accuracy_threshold 1215.458583831787 type value manhattan_ap 69.51465499538298 type value manhattan_f1 70.56159420289853 type value manhattan_f1_threshold 1344.7942733764648 type value manhattan_precision 61.77636796193497 type value manhattan_recall 82.259767687434 type value max_ap 69.83287846115917 type value max_f1 71.33520074696546 type value max_precision 63.93305439330545 type value max_recall 82.259767687434 type value similarity_accuracy 66.8651867893882 type value similarity_accuracy_threshold 84.34688448905945 type value similarity_ap 69.83287846115917 type value similarity_f1 71.33520074696546 type value similarity_f1_threshold 83.85992050170898 type value similarity_precision 63.93305439330545 type value similarity_recall 80.67581837381204 type PairClassification dataset metrics task config name revision split type default MTEB OnlineShopping e610f2ebd179a8fda30ae534c3878750a96db120 test C-MTEB/OnlineShopping-classification type value accuracy 93.66999999999999 type value ap 92.68160375501351 type value ap_weighted 92.68160375501351 type value f1 93.6673524115384 type value f1_weighted 93.67269842799493 type value main_score 93.66999999999999 type Classification dataset metrics task config name revision split type default MTEB PAWSX 9c6a90e430ac22b5779fb019a23e820b11a8b5e1 test C-MTEB/PAWSX type value cosine_pearson 14.427978400689973 type value cosine_spearman 15.182736434509348 type value euclidean_pearson 17.726048874983753 type value euclidean_spearman 15.201779286945575 type value main_score 15.182736434509348 type value manhattan_pearson 17.715716154164234 type value manhattan_spearman 15.250986981738777 type value pearson 14.427978400689973 type value spearman 15.182736434509348 type STS dataset metrics task config name revision split type default MTEB QBQTC 790b0510dc52b1553e8c49f3d2afb48c0e5c48b7 test C-MTEB/QBQTC type value cosine_pearson 28.677852039385687 type value cosine_spearman 30.317414500566187 type value euclidean_pearson 28.546943523039168 type value euclidean_spearman 30.31773442605619 type value main_score 30.317414500566187 type value manhattan_pearson 29.06524931618951 type value manhattan_spearman 30.85475318983088 type value pearson 28.677852039385687 type value spearman 30.317414500566187 type STS dataset metrics task config name revision split type zh MTEB STS22 (zh) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 74.25169654144152 type value cosine_spearman 74.02188505990078 type value euclidean_pearson 71.78459076777199 type value euclidean_spearman 74.02188505990078 type value main_score 74.02188505990078 type value manhattan_pearson 71.38471936226554 type value manhattan_spearman 73.72453020549669 type value pearson 74.25169654144152 type value spearman 74.02188505990078 type STS dataset metrics task config name revision split type zh-en MTEB STS22 (zh-en) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 77.21035942564082 type value cosine_spearman 76.57212143103963 type value euclidean_pearson 78.03973868360728 type value euclidean_spearman 76.57212143103963 type value main_score 76.57212143103963 type value manhattan_pearson 78.16591898142042 type value manhattan_spearman 76.83958214147293 type value pearson 77.21035942564082 type value spearman 76.57212143103963 type STS dataset metrics task config name revision split type default MTEB STSB 0cde68302b3541bb8b3c340dc0644b0b745b3dc0 test C-MTEB/STSB type value cosine_pearson 76.73366278962006 type value cosine_spearman 78.136597096582 type value euclidean_pearson 77.15227584574502 type value euclidean_spearman 78.13622498113003 type value main_score 78.136597096582 type value manhattan_pearson 77.02225035694117 type value manhattan_spearman 78.03964720563964 type value pearson 76.73366278962006 type value spearman 78.136597096582 type STS dataset metrics task config name revision split type default MTEB T2Reranking 76631901a18387f85eaa53e5450019b87ad58ef9 dev C-MTEB/T2Reranking type value main_score 66.38154648171584 type value map 66.38154648171584 type value mrr 76.14530606871499 type value nAUC_map_diff1 -9.806394737932642 type value nAUC_map_max 33.96115791248053 type value nAUC_map_std -3.643316859964786 type value nAUC_mrr_diff1 -6.510263484170889 type value nAUC_mrr_max 26.441557887574124 type value nAUC_mrr_std -4.608018494327204 type Reranking dataset metrics task config name revision split type default MTEB T2Retrieval 8731a845f1bf500a4f111cf1070785c793d10e64 dev C-MTEB/T2Retrieval type value main_score 84.133 type value map_at_1 27.297 type value map_at_10 76.494 type value map_at_100 80.119 type value map_at_1000 80.185 type value map_at_20 79.251 type value map_at_3 53.864999999999995 type value map_at_5 66.143 type value mrr_at_1 89.57566193231632 type value mrr_at_10 92.13000711126722 type value mrr_at_100 92.21882184581148 type value mrr_at_1000 92.22214774256558 type value mrr_at_20 92.18699134744894 type value mrr_at_3 91.66228300894257 type value mrr_at_5 91.97264597580231 type value nauc_map_at_1000_diff1 15.207460819974095 type value nauc_map_at_1000_max 42.32453165892631 type value nauc_map_at_1000_std 21.593634336302127 type value nauc_map_at_100_diff1 15.216272171820561 type value nauc_map_at_100_max 42.22983840076597 type value nauc_map_at_100_std 21.534370324932652 type value nauc_map_at_10_diff1 19.599553856210008 type value nauc_map_at_10_max 30.246318219245573 type value nauc_map_at_10_std 5.914404965156733 type value nauc_map_at_1_diff1 52.87085305237716 type value nauc_map_at_1_max -24.27989564325726 type value nauc_map_at_1_std -35.442050298290376 type value nauc_map_at_20_diff1 15.87998380728732 type value nauc_map_at_20_max 39.78308211411551 type value nauc_map_at_20_std 18.241218939315434 type value nauc_map_at_3_diff1 39.155089053329014 type value nauc_map_at_3_max -11.970155586820502 type value nauc_map_at_3_std -31.83333979404834 type value nauc_map_at_5_diff1 31.43539185744996 type value nauc_map_at_5_max 3.5586067754503152 type value nauc_map_at_5_std -20.89939723260621 type value nauc_mrr_at_1000_diff1 47.58856242843391 type value nauc_mrr_at_1000_max 73.33044542878086 type value nauc_mrr_at_1000_std 41.41370720044016 type value nauc_mrr_at_100_diff1 47.58885589082642 type value nauc_mrr_at_100_max 73.33895048178488 type value nauc_mrr_at_100_std 41.42862248729776 type value nauc_mrr_at_10_diff1 47.60432720674615 type value nauc_mrr_at_10_max 73.47964069672504 type value nauc_mrr_at_10_std 41.60604407817306 type value nauc_mrr_at_1_diff1 47.84195771830615 type value nauc_mrr_at_1_max 68.95221045759685 type value nauc_mrr_at_1_std 35.145250281429824 type value nauc_mrr_at_20_diff1 47.58534671931297 type value nauc_mrr_at_20_max 73.39618815713096 type value nauc_mrr_at_20_std 41.50538366605475 type value nauc_mrr_at_3_diff1 47.54080143480509 type value nauc_mrr_at_3_max 73.27456449852177 type value nauc_mrr_at_3_std 41.190010138623364 type value nauc_mrr_at_5_diff1 47.631799071300314 type value nauc_mrr_at_5_max 73.50427384392508 type value nauc_mrr_at_5_std 41.41445819292792 type value nauc_ndcg_at_1000_diff1 19.178203338132032 type value nauc_ndcg_at_1000_max 54.846002008332206 type value nauc_ndcg_at_1000_std 33.669755579706234 type value nauc_ndcg_at_100_diff1 18.825625578528154 type value nauc_ndcg_at_100_max 53.96154830438667 type value nauc_ndcg_at_100_std 33.63879617215427 type value nauc_ndcg_at_10_diff1 18.95559446945268 type value nauc_ndcg_at_10_max 44.21334528575739 type value nauc_ndcg_at_10_std 22.47737214494352 type value nauc_ndcg_at_1_diff1 47.84195771830615 type value nauc_ndcg_at_1_max 68.95221045759685 type value nauc_ndcg_at_1_std 35.145250281429824 type value nauc_ndcg_at_20_diff1 18.915787332802143 type value nauc_ndcg_at_20_max 48.64628634208606 type value nauc_ndcg_at_20_std 27.471901227649102 type value nauc_ndcg_at_3_diff1 14.800326460175548 type value nauc_ndcg_at_3_max 58.714123081214986 type value nauc_ndcg_at_3_std 32.87146819333138 type value nauc_ndcg_at_5_diff1 15.117887863548916 type value nauc_ndcg_at_5_max 51.62270126506565 type value nauc_ndcg_at_5_std 28.21637936542305 type value nauc_precision_at_1000_diff1 -34.6115257538737 type value nauc_precision_at_1000_max 46.57505454335497 type value nauc_precision_at_1000_std 58.73410354296305 type value nauc_precision_at_100_diff1 -34.51864090348213 type value nauc_precision_at_100_max 48.12778307352527 type value nauc_precision_at_100_std 60.33112526548986 type value nauc_precision_at_10_diff1 -33.913446995683536 type value nauc_precision_at_10_max 51.827800576762726 type value nauc_precision_at_10_std 56.15214316846719 type value nauc_precision_at_1_diff1 47.84195771830615 type value nauc_precision_at_1_max 68.95221045759685 type value nauc_precision_at_1_std 35.145250281429824 type value nauc_precision_at_20_diff1 -34.25535498799855 type value nauc_precision_at_20_max 50.23119733433027 type value nauc_precision_at_20_std 59.671418737988546 type value nauc_precision_at_3_diff1 -28.417107232598877 type value nauc_precision_at_3_max 61.16886341335774 type value nauc_precision_at_3_std 48.34533128391697 type value nauc_precision_at_5_diff1 -33.54570066440394 type value nauc_precision_at_5_max 56.522769824532936 type value nauc_precision_at_5_std 51.704950593707935 type value nauc_recall_at_1000_diff1 2.93977183499487 type value nauc_recall_at_1000_max 59.19161397622145 type value nauc_recall_at_1000_std 62.44563668374114 type value nauc_recall_at_100_diff1 8.013825549311562 type value nauc_recall_at_100_max 49.846341160862714 type value nauc_recall_at_100_std 48.1170998033127 type value nauc_recall_at_10_diff1 18.010735796887985 type value nauc_recall_at_10_max 21.358569425898903 type value nauc_recall_at_10_std 1.3301139186106035 type value nauc_recall_at_1_diff1 52.87085305237716 type value nauc_recall_at_1_max -24.27989564325726 type value nauc_recall_at_1_std -35.442050298290376 type value nauc_recall_at_20_diff1 11.816321531579238 type value nauc_recall_at_20_max 36.13782953010234 type value nauc_recall_at_20_std 23.555109581359886 type value nauc_recall_at_3_diff1 37.46336191367832 type value nauc_recall_at_3_max -16.038670342884316 type value nauc_recall_at_3_std -34.074784083025214 type value nauc_recall_at_5_diff1 30.274716744272567 type value nauc_recall_at_5_max -4.34067124108913 type value nauc_recall_at_5_std -26.21894992157237 type value ndcg_at_1 89.576 type value ndcg_at_10 84.133 type value ndcg_at_100 87.773 type value ndcg_at_1000 88.421 type value ndcg_at_20 85.909 type value ndcg_at_3 85.539 type value ndcg_at_5 84.143 type value precision_at_1 89.576 type value precision_at_10 41.789 type value precision_at_100 4.995 type value precision_at_1000 0.515 type value precision_at_20 23.224 type value precision_at_3 74.79400000000001 type value precision_at_5 62.683 type value recall_at_1 27.297 type value recall_at_10 83.035 type value recall_at_100 94.915 type value recall_at_1000 98.225 type value recall_at_20 88.984 type value recall_at_3 55.533 type value recall_at_5 69.575 type Retrieval dataset metrics task config name revision split type default MTEB TNews 317f262bf1e6126357bbe89e875451e4b0938fe4 validation C-MTEB/TNews-classification type value accuracy 51.664 type value f1 49.254634831292336 type value f1_weighted 51.23047453836118 type value main_score 51.664 type Classification dataset metrics task config name revision split type default MTEB ThuNewsClusteringP2P 5798586b105c0434e4f0fe5e767abe619442cf93 test C-MTEB/ThuNewsClusteringP2P type value main_score 62.931149356482294 type value v_measure 62.931149356482294 type value v_measure_std 1.2113879267357022 type Clustering dataset metrics task config name revision split type default MTEB ThuNewsClusteringS2S 8a8b2caeda43f39e13c4bc5bea0f8a667896e10d test C-MTEB/ThuNewsClusteringS2S type value main_score 59.18138500076393 type value v_measure 59.18138500076393 type value v_measure_std 1.441163494106974 type Clustering dataset metrics task config name revision split type default MTEB VideoRetrieval 58c2597a5943a2ba48f4668c3b90d796283c5639 dev C-MTEB/VideoRetrieval type value main_score 72.14500000000001 type value map_at_1 58.8 type value map_at_10 68.014 type value map_at_100 68.506 type value map_at_1000 68.51899999999999 type value map_at_20 68.333 type value map_at_3 66.31700000000001 type value map_at_5 67.31200000000001 type value mrr_at_1 58.8 type value mrr_at_10 68.01432539682544 type value mrr_at_100 68.50595347947811 type value mrr_at_1000 68.51919475199976 type value mrr_at_20 68.33299226014789 type value mrr_at_3 66.3166666666667 type value mrr_at_5 67.31166666666671 type value nauc_map_at_1000_diff1 68.3842603726721 type value nauc_map_at_1000_max 5.841784848188991 type value nauc_map_at_1000_std -31.890361063810364 type value nauc_map_at_100_diff1 68.38290538651279 type value nauc_map_at_100_max 5.851346250195991 type value nauc_map_at_100_std -31.88363804217233 type value nauc_map_at_10_diff1 68.42162270332948 type value nauc_map_at_10_max 5.545878771437991 type value nauc_map_at_10_std -32.33063386887081 type value nauc_map_at_1_diff1 69.28202470263717 type value nauc_map_at_1_max 5.263512365959786 type value nauc_map_at_1_std -29.659416343096055 type value nauc_map_at_20_diff1 68.3987969552634 type value nauc_map_at_20_max 5.7847092517499785 type value nauc_map_at_20_std -32.0616280955644 type value nauc_map_at_3_diff1 68.42478821018057 type value nauc_map_at_3_max 4.861120340503774 type value nauc_map_at_3_std -33.56938270962587 type value nauc_map_at_5_diff1 68.20507686427763 type value nauc_map_at_5_max 5.369798374942801 type value nauc_map_at_5_std -32.83081659270383 type value nauc_mrr_at_1000_diff1 68.3842603726721 type value nauc_mrr_at_1000_max 5.841784848188991 type value nauc_mrr_at_1000_std -31.890361063810364 type value nauc_mrr_at_100_diff1 68.38290538651279 type value nauc_mrr_at_100_max 5.851346250195991 type value nauc_mrr_at_100_std -31.88363804217233 type value nauc_mrr_at_10_diff1 68.42162270332948 type value nauc_mrr_at_10_max 5.545878771437991 type value nauc_mrr_at_10_std -32.33063386887081 type value nauc_mrr_at_1_diff1 69.28202470263717 type value nauc_mrr_at_1_max 5.263512365959786 type value nauc_mrr_at_1_std -29.659416343096055 type value nauc_mrr_at_20_diff1 68.3987969552634 type value nauc_mrr_at_20_max 5.7847092517499785 type value nauc_mrr_at_20_std -32.0616280955644 type value nauc_mrr_at_3_diff1 68.42478821018057 type value nauc_mrr_at_3_max 4.861120340503774 type value nauc_mrr_at_3_std -33.56938270962587 type value nauc_mrr_at_5_diff1 68.20507686427763 type value nauc_mrr_at_5_max 5.369798374942801 type value nauc_mrr_at_5_std -32.83081659270383 type value nauc_ndcg_at_1000_diff1 68.14552912036231 type value nauc_ndcg_at_1000_max 7.562355001802865 type value nauc_ndcg_at_1000_std -30.13999419402607 type value nauc_ndcg_at_100_diff1 68.09990028004812 type value nauc_ndcg_at_100_max 7.917285926128676 type value nauc_ndcg_at_100_std -29.909889861196902 type value nauc_ndcg_at_10_diff1 68.32387598538823 type value nauc_ndcg_at_10_max 6.442888130533218 type value nauc_ndcg_at_10_std -32.43505234576926 type value nauc_ndcg_at_1_diff1 69.28202470263717 type value nauc_ndcg_at_1_max 5.263512365959786 type value nauc_ndcg_at_1_std -29.659416343096055 type value nauc_ndcg_at_20_diff1 68.19058463118989 type value nauc_ndcg_at_20_max 7.4710128713487975 type value nauc_ndcg_at_20_std -31.212367402512527 type value nauc_ndcg_at_3_diff1 68.2422738747729 type value nauc_ndcg_at_3_max 4.866392479207864 type value nauc_ndcg_at_3_std -35.0611297009806 type value nauc_ndcg_at_5_diff1 67.76867006392196 type value nauc_ndcg_at_5_max 5.876702580928499 type value nauc_ndcg_at_5_std -33.66450752679279 type value nauc_precision_at_1000_diff1 59.01318860877509 type value nauc_precision_at_1000_max 92.88340336134347 type value nauc_precision_at_1000_std 92.92425303454743 type value nauc_precision_at_100_diff1 62.909039584826274 type value nauc_precision_at_100_max 53.748941437039655 type value nauc_precision_at_100_std 25.24916943521579 type value nauc_precision_at_10_diff1 68.09729905629663 type value nauc_precision_at_10_max 12.03384315001613 type value nauc_precision_at_10_std -31.81483891962282 type value nauc_precision_at_1_diff1 69.28202470263717 type value nauc_precision_at_1_max 5.263512365959786 type value nauc_precision_at_1_std -29.659416343096055 type value nauc_precision_at_20_diff1 66.6897634037554 type value nauc_precision_at_20_max 23.11402140195658 type value nauc_precision_at_20_std -20.564049852242167 type value nauc_precision_at_3_diff1 67.64170624528396 type value nauc_precision_at_3_max 4.945160628945999 type value nauc_precision_at_3_std -40.41499950328566 type value nauc_precision_at_5_diff1 65.92840910208848 type value nauc_precision_at_5_max 8.229706730154186 type value nauc_precision_at_5_std -36.74013989591443 type value nauc_recall_at_1000_diff1 59.01318860877662 type value nauc_recall_at_1000_max 92.88340336134418 type value nauc_recall_at_1000_std 92.92425303454706 type value nauc_recall_at_100_diff1 62.90903958482619 type value nauc_recall_at_100_max 53.748941437040145 type value nauc_recall_at_100_std 25.249169435216018 type value nauc_recall_at_10_diff1 68.0972990562968 type value nauc_recall_at_10_max 12.033843150016319 type value nauc_recall_at_10_std -31.814838919622566 type value nauc_recall_at_1_diff1 69.28202470263717 type value nauc_recall_at_1_max 5.263512365959786 type value nauc_recall_at_1_std -29.659416343096055 type value nauc_recall_at_20_diff1 66.6897634037554 type value nauc_recall_at_20_max 23.114021401956656 type value nauc_recall_at_20_std -20.564049852241986 type value nauc_recall_at_3_diff1 67.64170624528384 type value nauc_recall_at_3_max 4.9451606289460095 type value nauc_recall_at_3_std -40.41499950328563 type value nauc_recall_at_5_diff1 65.92840910208865 type value nauc_recall_at_5_max 8.229706730154424 type value nauc_recall_at_5_std -36.740139895914325 type value ndcg_at_1 58.8 type value ndcg_at_10 72.14500000000001 type value ndcg_at_100 74.477 type value ndcg_at_1000 74.821 type value ndcg_at_20 73.34 type value ndcg_at_3 68.634 type value ndcg_at_5 70.416 type value precision_at_1 58.8 type value precision_at_10 8.5 type value precision_at_100 0.9570000000000001 type value precision_at_1000 0.098 type value precision_at_20 4.49 type value precision_at_3 25.1 type value precision_at_5 15.920000000000002 type value recall_at_1 58.8 type value recall_at_10 85.0 type value recall_at_100 95.7 type value recall_at_1000 98.4 type value recall_at_20 89.8 type value recall_at_3 75.3 type value recall_at_5 79.60000000000001 type Retrieval dataset metrics task config name revision split type default MTEB Waimai 339287def212450dcaa9df8c22bf93e9980c7023 test C-MTEB/waimai-classification type value accuracy 89.12 type value ap 74.85094489946682 type value ap_weighted 74.85094489946682 type value f1 87.58964139879481 type value f1_weighted 89.11267843686537 type value main_score 89.12 type Classification dataset metrics task config name revision split type default MTEB AlloProfClusteringP2P 392ba3f5bcc8c51f578786c1fc3dae648662cb9b test lyon-nlp/alloprof type value main_score 66.7100274116735 type value v_measure 66.7100274116735 type value v_measure_std 2.065600197695283 type Clustering dataset metrics task config name revision split type default MTEB AlloProfClusteringS2S 392ba3f5bcc8c51f578786c1fc3dae648662cb9b test lyon-nlp/alloprof type value main_score 47.67572024379311 type value v_measure 47.67572024379311 type value v_measure_std 3.1905282169494953 type Clustering dataset metrics task config name revision split type default MTEB AlloprofReranking 65393d0d7a08a10b4e348135e824f385d420b0fd test lyon-nlp/mteb-fr-reranking-alloprof-s2p type value main_score 75.04647907753767 type value map 75.04647907753767 type value mrr 76.25801875154207 type value nAUC_map_diff1 56.38279442235466 type value nAUC_map_max 20.009630947768642 type value nAUC_map_std 21.626818227466185 type value nAUC_mrr_diff1 56.33463291672874 type value nAUC_mrr_max 20.472794140230853 type value nAUC_mrr_std 21.491759650866392 type Reranking dataset metrics task config name revision split type default MTEB AlloprofRetrieval fcf295ea64c750f41fadbaa37b9b861558e1bfbd test lyon-nlp/alloprof type value main_score 50.638000000000005 type value map_at_1 33.161 type value map_at_10 44.698 type value map_at_100 45.596 type value map_at_1000 45.635999999999996 type value map_at_20 45.265 type value map_at_3 41.703 type value map_at_5 43.488 type value mrr_at_1 33.160621761658035 type value mrr_at_10 44.697771883652734 type value mrr_at_100 45.59624815182174 type value mrr_at_1000 45.63609361771601 type value mrr_at_20 45.26480516767501 type value mrr_at_3 41.70264824409908 type value mrr_at_5 43.488054116292574 type value nauc_map_at_1000_diff1 38.49809004106204 type value nauc_map_at_1000_max 31.640827883359986 type value nauc_map_at_1000_std 2.5944833693677563 type value nauc_map_at_100_diff1 38.47974017961114 type value nauc_map_at_100_max 31.6745580307424 type value nauc_map_at_100_std 2.6197669693649965 type value nauc_map_at_10_diff1 38.43029274269754 type value nauc_map_at_10_max 31.669351274164402 type value nauc_map_at_10_std 2.2938216424530955 type value nauc_map_at_1_diff1 42.39449280665502 type value nauc_map_at_1_max 27.396202491464315 type value nauc_map_at_1_std 0.39154393747181304 type value nauc_map_at_20_diff1 38.44710465218088 type value nauc_map_at_20_max 31.618626111686442 type value nauc_map_at_20_std 2.5092690901463994 type value nauc_map_at_3_diff1 38.68180058655341 type value nauc_map_at_3_max 30.48704606797293 type value nauc_map_at_3_std 1.6764325554613773 type value nauc_map_at_5_diff1 38.27528363570654 type value nauc_map_at_5_max 31.105696409714735 type value nauc_map_at_5_std 2.3132867223174043 type value nauc_mrr_at_1000_diff1 38.49809004106204 type value nauc_mrr_at_1000_max 31.640827883359986 type value nauc_mrr_at_1000_std 2.5944833693677563 type value nauc_mrr_at_100_diff1 38.47974017961114 type value nauc_mrr_at_100_max 31.6745580307424 type value nauc_mrr_at_100_std 2.6197669693649965 type value nauc_mrr_at_10_diff1 38.43029274269754 type value nauc_mrr_at_10_max 31.669351274164402 type value nauc_mrr_at_10_std 2.2938216424530955 type value nauc_mrr_at_1_diff1 42.39449280665502 type value nauc_mrr_at_1_max 27.396202491464315 type value nauc_mrr_at_1_std 0.39154393747181304 type value nauc_mrr_at_20_diff1 38.44710465218088 type value nauc_mrr_at_20_max 31.618626111686442 type value nauc_mrr_at_20_std 2.5092690901463994 type value nauc_mrr_at_3_diff1 38.68180058655341 type value nauc_mrr_at_3_max 30.48704606797293 type value nauc_mrr_at_3_std 1.6764325554613773 type value nauc_mrr_at_5_diff1 38.27528363570654 type value nauc_mrr_at_5_max 31.105696409714735 type value nauc_mrr_at_5_std 2.3132867223174043 type value nauc_ndcg_at_1000_diff1 37.94639112622322 type value nauc_ndcg_at_1000_max 33.25000406312992 type value nauc_ndcg_at_1000_std 3.927246572224288 type value nauc_ndcg_at_100_diff1 37.488139235799 type value nauc_ndcg_at_100_max 34.38011697151766 type value nauc_ndcg_at_100_std 4.94760159362139 type value nauc_ndcg_at_10_diff1 37.318669958427996 type value nauc_ndcg_at_10_max 34.19162673981376 type value nauc_ndcg_at_10_std 3.2011892955083256 type value nauc_ndcg_at_1_diff1 42.39449280665502 type value nauc_ndcg_at_1_max 27.396202491464315 type value nauc_ndcg_at_1_std 0.39154393747181304 type value nauc_ndcg_at_20_diff1 37.290108058390985 type value nauc_ndcg_at_20_max 34.108858641349556 type value nauc_ndcg_at_20_std 4.169459504439506 type value nauc_ndcg_at_3_diff1 37.62224828453568 type value nauc_ndcg_at_3_max 31.519305313909218 type value nauc_ndcg_at_3_std 2.087339522812091 type value nauc_ndcg_at_5_diff1 36.888334499663785 type value nauc_ndcg_at_5_max 32.6601407781184 type value nauc_ndcg_at_5_std 3.2124484680546175 type value nauc_precision_at_1000_diff1 59.19282490825572 type value nauc_precision_at_1000_max 68.32089152822621 type value nauc_precision_at_1000_std 67.77161809421868 type value nauc_precision_at_100_diff1 29.47575945272322 type value nauc_precision_at_100_max 63.42195725833949 type value nauc_precision_at_100_std 34.923105379547344 type value nauc_precision_at_10_diff1 33.52501919318297 type value nauc_precision_at_10_max 44.49893440034256 type value nauc_precision_at_10_std 6.680426129369459 type value nauc_precision_at_1_diff1 42.39449280665502 type value nauc_precision_at_1_max 27.396202491464315 type value nauc_precision_at_1_std 0.39154393747181304 type value nauc_precision_at_20_diff1 32.17682672599943 type value nauc_precision_at_20_max 46.87049521936974 type value nauc_precision_at_20_std 13.53258332473726 type value nauc_precision_at_3_diff1 34.54132207851944 type value nauc_precision_at_3_max 34.574775459010255 type value nauc_precision_at_3_std 3.298031208443393 type value nauc_precision_at_5_diff1 32.475852196639195 type value nauc_precision_at_5_max 37.73978486643185 type value nauc_precision_at_5_std 6.185472179658329 type value nauc_recall_at_1000_diff1 59.19282490825427 type value nauc_recall_at_1000_max 68.32089152822542 type value nauc_recall_at_1000_std 67.77161809421989 type value nauc_recall_at_100_diff1 29.475759452723388 type value nauc_recall_at_100_max 63.421957258339425 type value nauc_recall_at_100_std 34.92310537954746 type value nauc_recall_at_10_diff1 33.525019193182956 type value nauc_recall_at_10_max 44.498934400342485 type value nauc_recall_at_10_std 6.680426129369434 type value nauc_recall_at_1_diff1 42.39449280665502 type value nauc_recall_at_1_max 27.396202491464315 type value nauc_recall_at_1_std 0.39154393747181304 type value nauc_recall_at_20_diff1 32.17682672599945 type value nauc_recall_at_20_max 46.87049521936974 type value nauc_recall_at_20_std 13.53258332473721 type value nauc_recall_at_3_diff1 34.54132207851946 type value nauc_recall_at_3_max 34.5747754590102 type value nauc_recall_at_3_std 3.2980312084433936 type value nauc_recall_at_5_diff1 32.47585219663912 type value nauc_recall_at_5_max 37.73978486643183 type value nauc_recall_at_5_std 6.18547217965832 type value ndcg_at_1 33.161 type value ndcg_at_10 50.638000000000005 type value ndcg_at_100 55.076 type value ndcg_at_1000 56.18300000000001 type value ndcg_at_20 52.681 type value ndcg_at_3 44.488 type value ndcg_at_5 47.705999999999996 type value precision_at_1 33.161 type value precision_at_10 6.9430000000000005 type value precision_at_100 0.9039999999999999 type value precision_at_1000 0.099 type value precision_at_20 3.873 type value precision_at_3 17.516000000000002 type value precision_at_5 12.073 type value recall_at_1 33.161 type value recall_at_10 69.43 type value recall_at_100 90.371 type value recall_at_1000 99.18 type value recall_at_20 77.461 type value recall_at_3 52.547 type value recall_at_5 60.363 type Retrieval dataset metrics task config name revision split type fr MTEB AmazonReviewsClassification (fr) 1399c76144fd37290681b995c656ef9b2e06e26d test mteb/amazon_reviews_multi type value accuracy 52.622 type value f1 48.89589865194384 type value f1_weighted 48.89589865194384 type value main_score 52.622 type Classification dataset metrics task config name revision split type default MTEB BSARDRetrieval 5effa1b9b5fa3b0f9e12523e6e43e5f86a6e6d59 test maastrichtlawtech/bsard type value main_score 59.458999999999996 type value map_at_1 8.559 type value map_at_10 15.853 type value map_at_100 16.955000000000002 type value map_at_1000 17.039 type value map_at_20 16.491 type value map_at_3 13.739 type value map_at_5 14.887 type value mrr_at_1 8.558558558558559 type value mrr_at_10 15.852995852995852 type value mrr_at_100 16.95536191852861 type value mrr_at_1000 17.03894008776081 type value mrr_at_20 16.490710101391212 type value mrr_at_3 13.738738738738734 type value mrr_at_5 14.887387387387385 type value nauc_map_at_1000_diff1 22.6427616709538 type value nauc_map_at_1000_max 30.273021433334108 type value nauc_map_at_1000_std 8.648862859092157 type value nauc_map_at_100_diff1 22.523593314805954 type value nauc_map_at_100_max 30.197098780769366 type value nauc_map_at_100_std 8.638222954134465 type value nauc_map_at_10_diff1 22.382593376046035 type value nauc_map_at_10_max 30.80647774104949 type value nauc_map_at_10_std 7.6451773140303825 type value nauc_map_at_1_diff1 32.27835486300824 type value nauc_map_at_1_max 31.839925744574 type value nauc_map_at_1_std 7.524965617228806 type value nauc_map_at_20_diff1 22.78130766181537 type value nauc_map_at_20_max 30.207832515412452 type value nauc_map_at_20_std 7.988030006241385 type value nauc_map_at_3_diff1 21.54291029527254 type value nauc_map_at_3_max 30.60738044134162 type value nauc_map_at_3_std 5.115709517278456 type value nauc_map_at_5_diff1 22.891658440504543 type value nauc_map_at_5_max 30.40238430175482 type value nauc_map_at_5_std 6.496264144977426 type value nauc_mrr_at_1000_diff1 22.6427616709538 type value nauc_mrr_at_1000_max 30.273021433334108 type value nauc_mrr_at_1000_std 8.648862859092157 type value nauc_mrr_at_100_diff1 22.523593314805954 type value nauc_mrr_at_100_max 30.197098780769366 type value nauc_mrr_at_100_std 8.638222954134465 type value nauc_mrr_at_10_diff1 22.382593376046035 type value nauc_mrr_at_10_max 30.80647774104949 type value nauc_mrr_at_10_std 7.6451773140303825 type value nauc_mrr_at_1_diff1 32.27835486300824 type value nauc_mrr_at_1_max 31.839925744574 type value nauc_mrr_at_1_std 7.524965617228806 type value nauc_mrr_at_20_diff1 22.78130766181537 type value nauc_mrr_at_20_max 30.207832515412452 type value nauc_mrr_at_20_std 7.988030006241385 type value nauc_mrr_at_3_diff1 21.54291029527254 type value nauc_mrr_at_3_max 30.60738044134162 type value nauc_mrr_at_3_std 5.115709517278456 type value nauc_mrr_at_5_diff1 22.891658440504543 type value nauc_mrr_at_5_max 30.40238430175482 type value nauc_mrr_at_5_std 6.496264144977426 type value nauc_ndcg_at_1000_diff1 22.131590111018863 type value nauc_ndcg_at_1000_max 30.119495176526417 type value nauc_ndcg_at_1000_std 14.152746889343884 type value nauc_ndcg_at_100_diff1 19.59019307197614 type value nauc_ndcg_at_100_max 29.26698074164439 type value nauc_ndcg_at_100_std 14.64843229218199 type value nauc_ndcg_at_10_diff1 20.04399986794229 type value nauc_ndcg_at_10_max 30.370494010101606 type value nauc_ndcg_at_10_std 9.076324266988427 type value nauc_ndcg_at_1_diff1 32.27835486300824 type value nauc_ndcg_at_1_max 31.839925744574 type value nauc_ndcg_at_1_std 7.524965617228806 type value nauc_ndcg_at_20_diff1 21.047174465558204 type value nauc_ndcg_at_20_max 28.383850745017487 type value nauc_ndcg_at_20_std 10.079085665060253 type value nauc_ndcg_at_3_diff1 18.696202337264843 type value nauc_ndcg_at_3_max 29.95559912145818 type value nauc_ndcg_at_3_std 4.515594333379446 type value nauc_ndcg_at_5_diff1 21.14710675076888 type value nauc_ndcg_at_5_max 29.60877022537729 type value nauc_ndcg_at_5_std 6.721635773882387 type value nauc_precision_at_1000_diff1 30.982325786968197 type value nauc_precision_at_1000_max 34.26481840304951 type value nauc_precision_at_1000_std 43.39003460634655 type value nauc_precision_at_100_diff1 11.987279247967425 type value nauc_precision_at_100_max 28.50285582800895 type value nauc_precision_at_100_std 35.49648389671331 type value nauc_precision_at_10_diff1 15.562900584507142 type value nauc_precision_at_10_max 29.558066061869663 type value nauc_precision_at_10_std 12.47595674036553 type value nauc_precision_at_1_diff1 32.27835486300824 type value nauc_precision_at_1_max 31.839925744574 type value nauc_precision_at_1_std 7.524965617228806 type value nauc_precision_at_20_diff1 18.081035071003427 type value nauc_precision_at_20_max 23.85063262716287 type value nauc_precision_at_20_std 15.071481920870877 type value nauc_precision_at_3_diff1 12.597351208698534 type value nauc_precision_at_3_max 28.496818992459538 type value nauc_precision_at_3_std 3.2373330095471893 type value nauc_precision_at_5_diff1 17.904830065631092 type value nauc_precision_at_5_max 27.89909851354525 type value nauc_precision_at_5_std 7.3432451499420734 type value nauc_recall_at_1000_diff1 30.982325786968097 type value nauc_recall_at_1000_max 34.264818403049496 type value nauc_recall_at_1000_std 43.39003460634647 type value nauc_recall_at_100_diff1 11.987279247967388 type value nauc_recall_at_100_max 28.502855828008883 type value nauc_recall_at_100_std 35.49648389671325 type value nauc_recall_at_10_diff1 15.562900584507085 type value nauc_recall_at_10_max 29.558066061869624 type value nauc_recall_at_10_std 12.475956740365447 type value nauc_recall_at_1_diff1 32.27835486300824 type value nauc_recall_at_1_max 31.839925744574 type value nauc_recall_at_1_std 7.524965617228806 type value nauc_recall_at_20_diff1 18.081035071003342 type value nauc_recall_at_20_max 23.850632627162785 type value nauc_recall_at_20_std 15.071481920870786 type value nauc_recall_at_3_diff1 12.597351208698562 type value nauc_recall_at_3_max 28.496818992459545 type value nauc_recall_at_3_std 3.237333009547214 type value nauc_recall_at_5_diff1 17.90483006563107 type value nauc_recall_at_5_max 27.89909851354522 type value nauc_recall_at_5_std 7.343245149942006 type value ndcg_at_1 8.559 type value ndcg_at_10 19.828000000000003 type value ndcg_at_100 25.468000000000004 type value ndcg_at_1000 28.058 type value ndcg_at_20 22.122 type value ndcg_at_3 15.524 type value ndcg_at_5 17.579 type value precision_at_1 8.559 type value precision_at_10 3.243 type value precision_at_100 0.5950000000000001 type value precision_at_1000 0.08099999999999999 type value precision_at_20 2.072 type value precision_at_3 6.907000000000001 type value precision_at_5 5.135 type value recall_at_1 8.559 type value recall_at_10 32.432 type value recall_at_100 59.458999999999996 type value recall_at_1000 80.631 type value recall_at_20 41.441 type value recall_at_3 20.721 type value recall_at_5 25.676 type Retrieval dataset metrics task config name revision split type default MTEB HALClusteringS2S e06ebbbb123f8144bef1a5d18796f3dec9ae2915 test lyon-nlp/clustering-hal-s2s type value main_score 26.958035381361377 type value v_measure 26.958035381361377 type value v_measure_std 2.401353383071989 type Clustering dataset metrics task config name revision split type fr MTEB MLSUMClusteringP2P (fr) b5d54f8f3b61ae17845046286940f03c6bc79bc7 test reciTAL/mlsum type value main_score 46.15554988136895 type value v_measure 46.15554988136895 type value v_measure_std 2.459531525134688 type Clustering dataset metrics task config name revision split type fr MTEB MLSUMClusteringS2S (fr) b5d54f8f3b61ae17845046286940f03c6bc79bc7 test reciTAL/mlsum type value main_score 45.73187202144909 type value v_measure 45.73187202144909 type value v_measure_std 1.6402520163270633 type Clustering dataset metrics task config name revision split type fr MTEB MTOPDomainClassification (fr) d80d48c1eb48d3562165c59d59d0034df9fff0bf test mteb/mtop_domain type value accuracy 95.78766050735986 type value f1 95.61497706645892 type value f1_weighted 95.79887587161483 type value main_score 95.78766050735986 type Classification dataset metrics task config name revision split type fr MTEB MTOPIntentClassification (fr) ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba test mteb/mtop_intent type value accuracy 80.8800501096148 type value f1 53.9945274705194 type value f1_weighted 80.94438738414857 type value main_score 80.8800501096148 type Classification dataset metrics task config name revision split type fra MTEB MasakhaNEWSClassification (fra) 18193f187b92da67168c655c9973a165ed9593dd test mteb/masakhanews type value accuracy 83.6255924170616 type value f1 79.70294641135138 type value f1_weighted 83.33457992982105 type value main_score 83.6255924170616 type Classification dataset metrics task config name revision split type fra MTEB MasakhaNEWSClusteringP2P (fra) 8ccc72e69e65f40c70e117d8b3c08306bb788b60 test masakhane/masakhanews type value main_score 77.1970570860131 type value v_measure 77.1970570860131 type value v_measure_std 22.0055550035463 type Clustering dataset metrics task config name revision split type fra MTEB MasakhaNEWSClusteringS2S (fra) 8ccc72e69e65f40c70e117d8b3c08306bb788b60 test masakhane/masakhanews type value main_score 65.92601417312947 type value v_measure 65.92601417312947 type value v_measure_std 30.421071440935687 type Clustering dataset metrics task config name revision split type fr MTEB MassiveIntentClassification (fr) 4672e20407010da34463acc759c162ca9734bca6 test mteb/amazon_massive_intent type value accuracy 69.5359784801614 type value f1 64.640488940591 type value f1_weighted 67.85916565361048 type value main_score 69.5359784801614 type Classification dataset metrics task config name revision split type fr MTEB MassiveScenarioClassification (fr) fad2c6e8459f9e1c45d9315f4953d921437d70f8 test mteb/amazon_massive_scenario type value accuracy 78.52723604572965 type value f1 77.1995224144067 type value f1_weighted 78.1215987283123 type value main_score 78.52723604572965 type Classification dataset metrics task config name revision split type fr MTEB MintakaRetrieval (fr) efa78cc2f74bbcd21eff2261f9e13aebe40b814e test jinaai/mintakaqa type value main_score 26.448 type value map_at_1 14.947 type value map_at_10 22.303 type value map_at_100 23.477999999999998 type value map_at_1000 23.586 type value map_at_20 22.962 type value map_at_3 19.949 type value map_at_5 21.252 type value mrr_at_1 14.946764946764945 type value mrr_at_10 22.303001053001033 type value mrr_at_100 23.478040499941816 type value mrr_at_1000 23.585987565381252 type value mrr_at_20 22.96198948271138 type value mrr_at_3 19.949494949494913 type value mrr_at_5 21.251706251706192 type value nauc_map_at_1000_diff1 30.124123232611005 type value nauc_map_at_1000_max 19.329718056410893 type value nauc_map_at_1000_std 3.7304142418877606 type value nauc_map_at_100_diff1 30.06763654065989 type value nauc_map_at_100_max 19.339926348634375 type value nauc_map_at_100_std 3.7507886962889376 type value nauc_map_at_10_diff1 30.235621359267817 type value nauc_map_at_10_max 19.315231135265865 type value nauc_map_at_10_std 3.888262415552999 type value nauc_map_at_1_diff1 37.87356036243269 type value nauc_map_at_1_max 17.63892349776284 type value nauc_map_at_1_std -2.0575597858386208 type value nauc_map_at_20_diff1 30.06800385756772 type value nauc_map_at_20_max 19.172804564418264 type value nauc_map_at_20_std 3.721149536049358 type value nauc_map_at_3_diff1 32.09160567273595 type value nauc_map_at_3_max 19.055280691204825 type value nauc_map_at_3_std 1.9160849079572526 type value nauc_map_at_5_diff1 30.81034541116131 type value nauc_map_at_5_max 19.172166581396308 type value nauc_map_at_5_std 3.251197681984862 type value nauc_mrr_at_1000_diff1 30.12412337741088 type value nauc_mrr_at_1000_max 19.329717809214035 type value nauc_mrr_at_1000_std 3.730414425912248 type value nauc_mrr_at_100_diff1 30.06763654065989 type value nauc_mrr_at_100_max 19.339926348634375 type value nauc_mrr_at_100_std 3.7507886962889376 type value nauc_mrr_at_10_diff1 30.235621359267817 type value nauc_mrr_at_10_max 19.315231135265865 type value nauc_mrr_at_10_std 3.888262415552999 type value nauc_mrr_at_1_diff1 37.87356036243269 type value nauc_mrr_at_1_max 17.63892349776284 type value nauc_mrr_at_1_std -2.0575597858386208 type value nauc_mrr_at_20_diff1 30.06800385756772 type value nauc_mrr_at_20_max 19.172804564418264 type value nauc_mrr_at_20_std 3.721149536049358 type value nauc_mrr_at_3_diff1 32.09160567273595 type value nauc_mrr_at_3_max 19.055280691204825 type value nauc_mrr_at_3_std 1.9160849079572526 type value nauc_mrr_at_5_diff1 30.81034541116131 type value nauc_mrr_at_5_max 19.172166581396308 type value nauc_mrr_at_5_std 3.251197681984862 type value nauc_ndcg_at_1000_diff1 28.057639637340476 type value nauc_ndcg_at_1000_max 20.172072747981893 type value nauc_ndcg_at_1000_std 5.991944827605241 type value nauc_ndcg_at_100_diff1 26.60019642442434 type value nauc_ndcg_at_100_max 20.47271103053784 type value nauc_ndcg_at_100_std 6.489412476969333 type value nauc_ndcg_at_10_diff1 27.165894912173762 type value nauc_ndcg_at_10_max 19.79447862928707 type value nauc_ndcg_at_10_std 6.648857204092722 type value nauc_ndcg_at_1_diff1 37.87356036243269 type value nauc_ndcg_at_1_max 17.63892349776284 type value nauc_ndcg_at_1_std -2.0575597858386208 type value nauc_ndcg_at_20_diff1 26.582793970516843 type value nauc_ndcg_at_20_max 19.348538329936638 type value nauc_ndcg_at_20_std 6.138040315782395 type value nauc_ndcg_at_3_diff1 30.57338000196413 type value nauc_ndcg_at_3_max 19.37852889877986 type value nauc_ndcg_at_3_std 3.0568087546329408 type value nauc_ndcg_at_5_diff1 28.469299405769632 type value nauc_ndcg_at_5_max 19.599386892314122 type value nauc_ndcg_at_5_std 5.299940395199246 type value nauc_precision_at_1000_diff1 24.170281200655943 type value nauc_precision_at_1000_max 39.623019898347664 type value nauc_precision_at_1000_std 44.81985014306762 type value nauc_precision_at_100_diff1 14.474857644755179 type value nauc_precision_at_100_max 26.05636850160609 type value nauc_precision_at_100_std 16.53010919038197 type value nauc_precision_at_10_diff1 19.584122367964167 type value nauc_precision_at_10_max 20.86686195708289 type value nauc_precision_at_10_std 13.525636908101404 type value nauc_precision_at_1_diff1 37.87356036243269 type value nauc_precision_at_1_max 17.63892349776284 type value nauc_precision_at_1_std -2.0575597858386208 type value nauc_precision_at_20_diff1 17.420915050615722 type value nauc_precision_at_20_max 19.45722509579383 type value nauc_precision_at_20_std 12.077196513907348 type value nauc_precision_at_3_diff1 26.865120457860247 type value nauc_precision_at_3_max 20.154933241021354 type value nauc_precision_at_3_std 5.86927947299836 type value nauc_precision_at_5_diff1 22.803351569123205 type value nauc_precision_at_5_max 20.623962388912666 type value nauc_precision_at_5_std 10.348629762758872 type value nauc_recall_at_1000_diff1 24.170281200656042 type value nauc_recall_at_1000_max 39.62301989834765 type value nauc_recall_at_1000_std 44.8198501430671 type value nauc_recall_at_100_diff1 14.474857644755254 type value nauc_recall_at_100_max 26.056368501606116 type value nauc_recall_at_100_std 16.530109190381985 type value nauc_recall_at_10_diff1 19.58412236796417 type value nauc_recall_at_10_max 20.866861957082875 type value nauc_recall_at_10_std 13.5256369081014 type value nauc_recall_at_1_diff1 37.87356036243269 type value nauc_recall_at_1_max 17.63892349776284 type value nauc_recall_at_1_std -2.0575597858386208 type value nauc_recall_at_20_diff1 17.420915050615708 type value nauc_recall_at_20_max 19.45722509579385 type value nauc_recall_at_20_std 12.077196513907353 type value nauc_recall_at_3_diff1 26.865120457860243 type value nauc_recall_at_3_max 20.15493324102137 type value nauc_recall_at_3_std 5.869279472998389 type value nauc_recall_at_5_diff1 22.803351569123215 type value nauc_recall_at_5_max 20.62396238891266 type value nauc_recall_at_5_std 10.348629762758849 type value ndcg_at_1 14.947 type value ndcg_at_10 26.448 type value ndcg_at_100 32.78 type value ndcg_at_1000 35.937000000000005 type value ndcg_at_20 28.842000000000002 type value ndcg_at_3 21.587999999999997 type value ndcg_at_5 23.942 type value precision_at_1 14.947 type value precision_at_10 3.972 type value precision_at_100 0.7080000000000001 type value precision_at_1000 0.096 type value precision_at_20 2.459 type value precision_at_3 8.777 type value precision_at_5 6.413 type value recall_at_1 14.947 type value recall_at_10 39.722 type value recall_at_100 70.844 type value recall_at_1000 96.274 type value recall_at_20 49.181000000000004 type value recall_at_3 26.331 type value recall_at_5 32.064 type Retrieval dataset metrics task config name revision split type fr MTEB OpusparcusPC (fr) 9e9b1f8ef51616073f47f306f7f47dd91663f86a test GEM/opusparcus type value cosine_accuracy 82.62942779291554 type value cosine_accuracy_threshold 83.4860622882843 type value cosine_ap 93.39616519364185 type value cosine_f1 88.03378695448146 type value cosine_f1_threshold 83.4860622882843 type value cosine_precision 83.45195729537367 type value cosine_recall 93.14796425024826 type value dot_accuracy 82.62942779291554 type value dot_accuracy_threshold 83.4860622882843 type value dot_ap 93.39616519364185 type value dot_f1 88.03378695448146 type value dot_f1_threshold 83.4860622882843 type value dot_precision 83.45195729537367 type value dot_recall 93.14796425024826 type value euclidean_accuracy 82.62942779291554 type value euclidean_accuracy_threshold 57.4698805809021 type value euclidean_ap 93.39616519364185 type value euclidean_f1 88.03378695448146 type value euclidean_f1_threshold 57.4698805809021 type value euclidean_precision 83.45195729537367 type value euclidean_recall 93.14796425024826 type value main_score 93.39616519364185 type value manhattan_accuracy 82.62942779291554 type value manhattan_accuracy_threshold 1306.7530632019043 type value manhattan_ap 93.34098710518775 type value manhattan_f1 87.78409090909089 type value manhattan_f1_threshold 1335.2685928344727 type value manhattan_precision 83.89140271493213 type value manhattan_recall 92.05561072492551 type value max_ap 93.39616519364185 type value max_f1 88.03378695448146 type value max_precision 83.89140271493213 type value max_recall 93.14796425024826 type value similarity_accuracy 82.62942779291554 type value similarity_accuracy_threshold 83.4860622882843 type value similarity_ap 93.39616519364185 type value similarity_f1 88.03378695448146 type value similarity_f1_threshold 83.4860622882843 type value similarity_precision 83.45195729537367 type value similarity_recall 93.14796425024826 type PairClassification dataset metrics task config name revision split type fr MTEB PawsXPairClassification (fr) 8a04d940a42cd40658986fdd8e3da561533a3646 test google-research-datasets/paws-x type value cosine_accuracy 60.8 type value cosine_accuracy_threshold 98.90193939208984 type value cosine_ap 60.50913122978733 type value cosine_f1 62.69411339833874 type value cosine_f1_threshold 95.17210125923157 type value cosine_precision 46.51661307609861 type value cosine_recall 96.12403100775194 type value dot_accuracy 60.8 type value dot_accuracy_threshold 98.9019513130188 type value dot_ap 60.49770725998639 type value dot_f1 62.69411339833874 type value dot_f1_threshold 95.17210721969604 type value dot_precision 46.51661307609861 type value dot_recall 96.12403100775194 type value euclidean_accuracy 60.8 type value euclidean_accuracy_threshold 14.819307625293732 type value euclidean_ap 60.50917425308617 type value euclidean_f1 62.69411339833874 type value euclidean_f1_threshold 31.07377290725708 type value euclidean_precision 46.51661307609861 type value euclidean_recall 96.12403100775194 type value main_score 60.73371250119265 type value manhattan_accuracy 60.9 type value manhattan_accuracy_threshold 354.8734188079834 type value manhattan_ap 60.73371250119265 type value manhattan_f1 62.70506744440393 type value manhattan_f1_threshold 711.578369140625 type value manhattan_precision 46.73913043478261 type value manhattan_recall 95.23809523809523 type value max_ap 60.73371250119265 type value max_f1 62.70506744440393 type value max_precision 46.73913043478261 type value max_recall 96.12403100775194 type value similarity_accuracy 60.8 type value similarity_accuracy_threshold 98.90193939208984 type value similarity_ap 60.50913122978733 type value similarity_f1 62.69411339833874 type value similarity_f1_threshold 95.17210125923157 type value similarity_precision 46.51661307609861 type value similarity_recall 96.12403100775194 type PairClassification dataset metrics task config name revision split type default MTEB SICKFr e077ab4cf4774a1e36d86d593b150422fafd8e8a test Lajavaness/SICK-fr type value cosine_pearson 81.02846310969592 type value cosine_spearman 77.47140335069184 type value euclidean_pearson 77.4818795209704 type value euclidean_spearman 77.4714043813526 type value main_score 77.47140335069184 type value manhattan_pearson 77.44622115854098 type value manhattan_spearman 77.29743297817558 type value pearson 81.02846310969592 type value spearman 77.47140335069184 type STS dataset metrics task config name revision split type fr MTEB STS22 (fr) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 77.1356210910051 type value cosine_spearman 81.7065039306575 type value euclidean_pearson 79.32575551712296 type value euclidean_spearman 81.75624482168821 type value main_score 81.7065039306575 type value manhattan_pearson 81.05436417153798 type value manhattan_spearman 82.13370902176736 type value pearson 77.1356210910051 type value spearman 81.7065039306575 type STS dataset metrics task config name revision split type de-fr MTEB STS22 (de-fr) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 61.40659325490285 type value cosine_spearman 64.21007088135842 type value euclidean_pearson 61.051174476106 type value euclidean_spearman 64.21007088135842 type value main_score 64.21007088135842 type value manhattan_pearson 60.225817072214525 type value manhattan_spearman 64.32288638294209 type value pearson 61.40659325490285 type value spearman 64.21007088135842 type STS dataset metrics task config name revision split type fr-pl MTEB STS22 (fr-pl) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 88.17138238483673 type value cosine_spearman 84.51542547285167 type value euclidean_pearson 87.99782696047525 type value euclidean_spearman 84.51542547285167 type value main_score 84.51542547285167 type value manhattan_pearson 85.811937669563 type value manhattan_spearman 84.51542547285167 type value pearson 88.17138238483673 type value spearman 84.51542547285167 type STS dataset metrics task config name revision split type fr MTEB STSBenchmarkMultilingualSTS (fr) 29afa2569dcedaaa2fe6a3dcfebab33d28b82e8c test mteb/stsb_multi_mt type value cosine_pearson 79.98375089796882 type value cosine_spearman 81.06570417849169 type value euclidean_pearson 79.44759787417051 type value euclidean_spearman 81.06430479357311 type value main_score 81.06570417849169 type value manhattan_pearson 79.34683573713086 type value manhattan_spearman 81.00584846124926 type value pearson 79.98375089796882 type value spearman 81.06570417849169 type STS dataset metrics task config name revision split type default MTEB SummEvalFr b385812de6a9577b6f4d0f88c6a6e35395a94054 test lyon-nlp/summarization-summeval-fr-p2p type value cosine_pearson 31.198220154029464 type value cosine_spearman 30.886000528607877 type value dot_pearson 31.19822718500702 type value dot_spearman 30.86590068433314 type value main_score 30.886000528607877 type value pearson 31.198220154029464 type value spearman 30.886000528607877 type Summarization dataset metrics task config name revision split type default MTEB SyntecReranking daf0863838cd9e3ba50544cdce3ac2b338a1b0ad test lyon-nlp/mteb-fr-reranking-syntec-s2p type value main_score 86.6 type value map 86.6 type value mrr 86.6 type value nAUC_map_diff1 59.66160008216082 type value nAUC_map_max 19.768885092568734 type value nAUC_map_std 44.66975354255961 type value nAUC_mrr_diff1 59.66160008216082 type value nAUC_mrr_max 19.768885092568734 type value nAUC_mrr_std 44.66975354255961 type Reranking dataset metrics task config name revision split type default MTEB SyntecRetrieval 19661ccdca4dfc2d15122d776b61685f48c68ca9 test lyon-nlp/mteb-fr-retrieval-syntec-s2p type value main_score 81.899 type value map_at_1 64.0 type value map_at_10 76.594 type value map_at_100 76.66199999999999 type value map_at_1000 76.66199999999999 type value map_at_20 76.644 type value map_at_3 74.833 type value map_at_5 76.183 type value mrr_at_1 64.0 type value mrr_at_10 76.59404761904761 type value mrr_at_100 76.66159147869675 type value mrr_at_1000 76.66159147869675 type value mrr_at_20 76.64404761904763 type value mrr_at_3 74.83333333333333 type value mrr_at_5 76.18333333333334 type value nauc_map_at_1000_diff1 53.82627007182553 type value nauc_map_at_1000_max 17.927045359651704 type value nauc_map_at_1000_std -6.973071195715382 type value nauc_map_at_100_diff1 53.82627007182553 type value nauc_map_at_100_max 17.927045359651704 type value nauc_map_at_100_std -6.973071195715382 type value nauc_map_at_10_diff1 53.90625505629818 type value nauc_map_at_10_max 18.12979815440444 type value nauc_map_at_10_std -6.664265062780913 type value nauc_map_at_1_diff1 57.671797164388835 type value nauc_map_at_1_max 16.9354323668412 type value nauc_map_at_1_std -11.064631498275675 type value nauc_map_at_20_diff1 53.789271077104125 type value nauc_map_at_20_max 17.922015037605867 type value nauc_map_at_20_std -6.934974465544576 type value nauc_map_at_3_diff1 52.10054809507078 type value nauc_map_at_3_max 17.282564201023686 type value nauc_map_at_3_std -7.316507696153171 type value nauc_map_at_5_diff1 53.84305456072319 type value nauc_map_at_5_max 18.0761340059772 type value nauc_map_at_5_std -6.788097105243701 type value nauc_mrr_at_1000_diff1 53.82627007182553 type value nauc_mrr_at_1000_max 17.927045359651704 type value nauc_mrr_at_1000_std -6.973071195715382 type value nauc_mrr_at_100_diff1 53.82627007182553 type value nauc_mrr_at_100_max 17.927045359651704 type value nauc_mrr_at_100_std -6.973071195715382 type value nauc_mrr_at_10_diff1 53.90625505629818 type value nauc_mrr_at_10_max 18.12979815440444 type value nauc_mrr_at_10_std -6.664265062780913 type value nauc_mrr_at_1_diff1 57.671797164388835 type value nauc_mrr_at_1_max 16.9354323668412 type value nauc_mrr_at_1_std -11.064631498275675 type value nauc_mrr_at_20_diff1 53.789271077104125 type value nauc_mrr_at_20_max 17.922015037605867 type value nauc_mrr_at_20_std -6.934974465544576 type value nauc_mrr_at_3_diff1 52.10054809507078 type value nauc_mrr_at_3_max 17.282564201023686 type value nauc_mrr_at_3_std -7.316507696153171 type value nauc_mrr_at_5_diff1 53.84305456072319 type value nauc_mrr_at_5_max 18.0761340059772 type value nauc_mrr_at_5_std -6.788097105243701 type value nauc_ndcg_at_1000_diff1 53.47773846493816 type value nauc_ndcg_at_1000_max 18.270810672735895 type value nauc_ndcg_at_1000_std -6.204392784046327 type value nauc_ndcg_at_100_diff1 53.47773846493816 type value nauc_ndcg_at_100_max 18.270810672735895 type value nauc_ndcg_at_100_std -6.204392784046327 type value nauc_ndcg_at_10_diff1 53.70897446254982 type value nauc_ndcg_at_10_max 19.41340528944212 type value nauc_ndcg_at_10_std -4.167245194562443 type value nauc_ndcg_at_1_diff1 57.671797164388835 type value nauc_ndcg_at_1_max 16.9354323668412 type value nauc_ndcg_at_1_std -11.064631498275675 type value nauc_ndcg_at_20_diff1 53.013882632385034 type value nauc_ndcg_at_20_max 18.20334171980294 type value nauc_ndcg_at_20_std -5.7313885736485455 type value nauc_ndcg_at_3_diff1 49.798853568516044 type value nauc_ndcg_at_3_max 17.88910440624622 type value nauc_ndcg_at_3_std -5.959252175174665 type value nauc_ndcg_at_5_diff1 53.565830685346896 type value nauc_ndcg_at_5_max 19.301209293805627 type value nauc_ndcg_at_5_std -4.5368156313357435 type value nauc_precision_at_1000_diff1 .nan type value nauc_precision_at_1000_max .nan type value nauc_precision_at_1000_std .nan type value nauc_precision_at_100_diff1 .nan type value nauc_precision_at_100_max .nan type value nauc_precision_at_100_std .nan type value nauc_precision_at_10_diff1 56.13912231559286 type value nauc_precision_at_10_max 56.13912231559286 type value nauc_precision_at_10_std 67.9038281979461 type value nauc_precision_at_1_diff1 57.671797164388835 type value nauc_precision_at_1_max 16.9354323668412 type value nauc_precision_at_1_std -11.064631498275675 type value nauc_precision_at_20_diff1 12.278244631185926 type value nauc_precision_at_20_max 12.278244631185926 type value nauc_precision_at_20_std 35.80765639589114 type value nauc_precision_at_3_diff1 36.90404604415416 type value nauc_precision_at_3_max 21.58749248346349 type value nauc_precision_at_3_std 1.5204879305900956 type value nauc_precision_at_5_diff1 53.47338935574264 type value nauc_precision_at_5_max 33.86554621848775 type value nauc_precision_at_5_std 22.00746965452886 type value nauc_recall_at_1000_diff1 .nan type value nauc_recall_at_1000_max .nan type value nauc_recall_at_1000_std .nan type value nauc_recall_at_100_diff1 .nan type value nauc_recall_at_100_max .nan type value nauc_recall_at_100_std .nan type value nauc_recall_at_10_diff1 56.13912231559305 type value nauc_recall_at_10_max 56.13912231559305 type value nauc_recall_at_10_std 67.903828197946 type value nauc_recall_at_1_diff1 57.671797164388835 type value nauc_recall_at_1_max 16.9354323668412 type value nauc_recall_at_1_std -11.064631498275675 type value nauc_recall_at_20_diff1 12.278244631185359 type value nauc_recall_at_20_max 12.278244631185359 type value nauc_recall_at_20_std 35.80765639589109 type value nauc_recall_at_3_diff1 36.904046044154384 type value nauc_recall_at_3_max 21.587492483463492 type value nauc_recall_at_3_std 1.5204879305901602 type value nauc_recall_at_5_diff1 53.47338935574226 type value nauc_recall_at_5_max 33.86554621848721 type value nauc_recall_at_5_std 22.00746965452852 type value ndcg_at_1 64.0 type value ndcg_at_10 81.899 type value ndcg_at_100 82.297 type value ndcg_at_1000 82.297 type value ndcg_at_20 82.126 type value ndcg_at_3 78.464 type value ndcg_at_5 80.917 type value precision_at_1 64.0 type value precision_at_10 9.8 type value precision_at_100 1.0 type value precision_at_1000 0.1 type value precision_at_20 4.95 type value precision_at_3 29.666999999999998 type value precision_at_5 19.0 type value recall_at_1 64.0 type value recall_at_10 98.0 type value recall_at_100 100.0 type value recall_at_1000 100.0 type value recall_at_20 99.0 type value recall_at_3 89.0 type value recall_at_5 95.0 type Retrieval dataset metrics task config name revision split type fra-fra MTEB XPQARetrieval (fr) c99d599f0a6ab9b85b065da6f9d94f9cf731679f test jinaai/xpqa type value main_score 62.09 type value map_at_1 36.073 type value map_at_10 55.677 type value map_at_100 57.118 type value map_at_1000 57.199 type value map_at_20 56.501999999999995 type value map_at_3 49.619 type value map_at_5 53.455 type value mrr_at_1 57.543391188251 type value mrr_at_10 65.30018861127007 type value mrr_at_100 65.94099315822325 type value mrr_at_1000 65.96453864135188 type value mrr_at_20 65.71224590825028 type value mrr_at_3 63.351134846461946 type value mrr_at_5 64.42590120160212 type value nauc_map_at_1000_diff1 50.58271523935874 type value nauc_map_at_1000_max 43.13816564852953 type value nauc_map_at_1000_std 0.1844114463791253 type value nauc_map_at_100_diff1 50.55402514007517 type value nauc_map_at_100_max 43.131135237384484 type value nauc_map_at_100_std 0.1962985407010382 type value nauc_map_at_10_diff1 50.211948710332386 type value nauc_map_at_10_max 42.56586858276775 type value nauc_map_at_10_std -0.21682461914908613 type value nauc_map_at_1_diff1 58.97288229253611 type value nauc_map_at_1_max 27.09256578748547 type value nauc_map_at_1_std -3.128360909366587 type value nauc_map_at_20_diff1 50.33687763446524 type value nauc_map_at_20_max 42.89316787999387 type value nauc_map_at_20_std -0.2224194056336769 type value nauc_map_at_3_diff1 51.23147801843447 type value nauc_map_at_3_max 37.22691523640508 type value nauc_map_at_3_std -1.4704387784247346 type value nauc_map_at_5_diff1 50.66157676518992 type value nauc_map_at_5_max 41.58957149577394 type value nauc_map_at_5_std 0.16909716462753255 type value nauc_mrr_at_1000_diff1 58.88383847534171 type value nauc_mrr_at_1000_max 49.3245365217643 type value nauc_mrr_at_1000_std 2.1575868813952894 type value nauc_mrr_at_100_diff1 58.885865820137276 type value nauc_mrr_at_100_max 49.32954909327622 type value nauc_mrr_at_100_std 2.1750885487117024 type value nauc_mrr_at_10_diff1 58.83761987066026 type value nauc_mrr_at_10_max 49.37803355533766 type value nauc_mrr_at_10_std 1.927477967317313 type value nauc_mrr_at_1_diff1 60.897823384674496 type value nauc_mrr_at_1_max 48.79303626218842 type value nauc_mrr_at_1_std 3.68732973455558 type value nauc_mrr_at_20_diff1 58.80334636154898 type value nauc_mrr_at_20_max 49.299926776474535 type value nauc_mrr_at_20_std 1.9599488796786029 type value nauc_mrr_at_3_diff1 59.21037240205004 type value nauc_mrr_at_3_max 49.14597672580709 type value nauc_mrr_at_3_std 1.0051764061328385 type value nauc_mrr_at_5_diff1 58.98849095570841 type value nauc_mrr_at_5_max 49.68364568027881 type value nauc_mrr_at_5_std 2.4739579029654366 type value nauc_ndcg_at_1000_diff1 52.31164533549997 type value nauc_ndcg_at_1000_max 45.69420989458311 type value nauc_ndcg_at_1000_std 1.1608489877596142 type value nauc_ndcg_at_100_diff1 51.87286842964108 type value nauc_ndcg_at_100_max 45.685834956792895 type value nauc_ndcg_at_100_std 1.8157949218428466 type value nauc_ndcg_at_10_diff1 50.57331251457611 type value nauc_ndcg_at_10_max 44.44795063905562 type value nauc_ndcg_at_10_std -0.3915488786381922 type value nauc_ndcg_at_1_diff1 60.897823384674496 type value nauc_ndcg_at_1_max 48.79303626218842 type value nauc_ndcg_at_1_std 3.68732973455558 type value nauc_ndcg_at_20_diff1 50.76487704699518 type value nauc_ndcg_at_20_max 44.79388134049559 type value nauc_ndcg_at_20_std -0.4213693889586553 type value nauc_ndcg_at_3_diff1 51.177774035828605 type value nauc_ndcg_at_3_max 43.73405047316084 type value nauc_ndcg_at_3_std -1.18104282095782 type value nauc_ndcg_at_5_diff1 51.15375930024702 type value nauc_ndcg_at_5_max 43.7940523142017 type value nauc_ndcg_at_5_std 0.8224796779269716 type value nauc_precision_at_1000_diff1 -13.700846719394837 type value nauc_precision_at_1000_max 15.005182092410575 type value nauc_precision_at_1000_std 6.913901876028514 type value nauc_precision_at_100_diff1 -8.919890455110265 type value nauc_precision_at_100_max 20.85944528699816 type value nauc_precision_at_100_std 8.934660613911344 type value nauc_precision_at_10_diff1 2.0626021976371662 type value nauc_precision_at_10_max 30.851331908454423 type value nauc_precision_at_10_std 4.512923316711585 type value nauc_precision_at_1_diff1 60.897823384674496 type value nauc_precision_at_1_max 48.79303626218842 type value nauc_precision_at_1_std 3.68732973455558 type value nauc_precision_at_20_diff1 -1.9918582602200585 type value nauc_precision_at_20_max 27.779932491338315 type value nauc_precision_at_20_std 4.734186088720616 type value nauc_precision_at_3_diff1 14.5090169489911 type value nauc_precision_at_3_max 37.59006778251299 type value nauc_precision_at_3_std 3.677659738072369 type value nauc_precision_at_5_diff1 7.705804886616575 type value nauc_precision_at_5_max 36.0216894270471 type value nauc_precision_at_5_std 6.513474617464925 type value nauc_recall_at_1000_diff1 20.71811619738829 type value nauc_recall_at_1000_max 23.217180195398225 type value nauc_recall_at_1000_std 26.037508089878237 type value nauc_recall_at_100_diff1 38.44958378050671 type value nauc_recall_at_100_max 40.99327582118083 type value nauc_recall_at_100_std 16.36015422588489 type value nauc_recall_at_10_diff1 40.027789080211576 type value nauc_recall_at_10_max 38.82613587358396 type value nauc_recall_at_10_std -3.5237192778606596 type value nauc_recall_at_1_diff1 58.97288229253611 type value nauc_recall_at_1_max 27.09256578748547 type value nauc_recall_at_1_std -3.128360909366587 type value nauc_recall_at_20_diff1 37.818919303571406 type value nauc_recall_at_20_max 37.42703966259237 type value nauc_recall_at_20_std -4.770317748130178 type value nauc_recall_at_3_diff1 45.13163472734054 type value nauc_recall_at_3_max 33.72267598718042 type value nauc_recall_at_3_std -4.443802840190085 type value nauc_recall_at_5_diff1 43.05114612174671 type value nauc_recall_at_5_max 39.10347802906311 type value nauc_recall_at_5_std 0.4813526343602913 type value ndcg_at_1 57.543 type value ndcg_at_10 62.09 type value ndcg_at_100 67.216 type value ndcg_at_1000 68.60000000000001 type value ndcg_at_20 64.20700000000001 type value ndcg_at_3 56.952999999999996 type value ndcg_at_5 58.631 type value precision_at_1 57.543 type value precision_at_10 14.499 type value precision_at_100 1.8739999999999999 type value precision_at_1000 0.20600000000000002 type value precision_at_20 7.971 type value precision_at_3 34.446 type value precision_at_5 24.993000000000002 type value recall_at_1 36.073 type value recall_at_10 70.532 type value recall_at_100 90.63600000000001 type value recall_at_1000 99.577 type value recall_at_20 77.388 type value recall_at_3 54.786 type value recall_at_5 62.365 type Retrieval dataset metrics task config name revision split type eng-fra MTEB XPQARetrieval (eng-fra) c99d599f0a6ab9b85b065da6f9d94f9cf731679f test jinaai/xpqa type value main_score 34.795 type value map_at_1 13.818 type value map_at_10 28.221 type value map_at_100 30.715999999999998 type value map_at_1000 30.86 type value map_at_20 29.601 type value map_at_3 23.194 type value map_at_5 26.057999999999996 type value mrr_at_1 27.236315086782376 type value mrr_at_10 36.39890224002375 type value mrr_at_100 37.73446796439471 type value mrr_at_1000 37.79021013088287 type value mrr_at_20 37.17175635350331 type value mrr_at_3 33.73386737872722 type value mrr_at_5 35.24922118380064 type value nauc_map_at_1000_diff1 32.30220782845437 type value nauc_map_at_1000_max 39.87665001530303 type value nauc_map_at_1000_std 5.7695221727058055 type value nauc_map_at_100_diff1 32.2694600306288 type value nauc_map_at_100_max 39.8855550981263 type value nauc_map_at_100_std 5.776881467271089 type value nauc_map_at_10_diff1 31.804039669931537 type value nauc_map_at_10_max 39.311719475930005 type value nauc_map_at_10_std 4.733050895784147 type value nauc_map_at_1_diff1 37.6388258626095 type value nauc_map_at_1_max 25.192760889147102 type value nauc_map_at_1_std 0.84012346712856 type value nauc_map_at_20_diff1 31.970587565845022 type value nauc_map_at_20_max 39.68812698793437 type value nauc_map_at_20_std 5.466710545588436 type value nauc_map_at_3_diff1 33.9083845702625 type value nauc_map_at_3_max 35.88443788757562 type value nauc_map_at_3_std 2.956590608487331 type value nauc_map_at_5_diff1 32.97116962607063 type value nauc_map_at_5_max 38.37964967819906 type value nauc_map_at_5_std 4.573297881379916 type value nauc_mrr_at_1000_diff1 32.44126725520097 type value nauc_mrr_at_1000_max 39.246349656799985 type value nauc_mrr_at_1000_std 7.779126542191918 type value nauc_mrr_at_100_diff1 32.416259735518885 type value nauc_mrr_at_100_max 39.258316221246965 type value nauc_mrr_at_100_std 7.7944505673136 type value nauc_mrr_at_10_diff1 32.06932803779604 type value nauc_mrr_at_10_max 39.04853109147614 type value nauc_mrr_at_10_std 7.4138965935269505 type value nauc_mrr_at_1_diff1 37.27173489316227 type value nauc_mrr_at_1_max 40.3355905491979 type value nauc_mrr_at_1_std 6.933728079474825 type value nauc_mrr_at_20_diff1 32.329474296004804 type value nauc_mrr_at_20_max 39.0764118546337 type value nauc_mrr_at_20_std 7.696441003623004 type value nauc_mrr_at_3_diff1 33.08673004752433 type value nauc_mrr_at_3_max 39.95985433324281 type value nauc_mrr_at_3_std 7.62764311528151 type value nauc_mrr_at_5_diff1 32.818965514653684 type value nauc_mrr_at_5_max 39.34493265770003 type value nauc_mrr_at_5_std 7.778531920242 type value nauc_ndcg_at_1000_diff1 31.02864530261756 type value nauc_ndcg_at_1000_max 39.64187241406462 type value nauc_ndcg_at_1000_std 7.768459120817835 type value nauc_ndcg_at_100_diff1 30.39095044516521 type value nauc_ndcg_at_100_max 39.956877555291406 type value nauc_ndcg_at_100_std 8.813305671545828 type value nauc_ndcg_at_10_diff1 29.09294115578835 type value nauc_ndcg_at_10_max 38.59245602933513 type value nauc_ndcg_at_10_std 5.516145701680656 type value nauc_ndcg_at_1_diff1 37.27173489316227 type value nauc_ndcg_at_1_max 40.3355905491979 type value nauc_ndcg_at_1_std 6.933728079474825 type value nauc_ndcg_at_20_diff1 29.725541536865684 type value nauc_ndcg_at_20_max 39.12781667827556 type value nauc_ndcg_at_20_std 7.464557759930056 type value nauc_ndcg_at_3_diff1 32.2472918241563 type value nauc_ndcg_at_3_max 39.38528978160266 type value nauc_ndcg_at_3_std 5.126228097274878 type value nauc_ndcg_at_5_diff1 31.39000117667687 type value nauc_ndcg_at_5_max 38.16838826710958 type value nauc_ndcg_at_5_std 5.747613838798295 type value nauc_precision_at_1000_diff1 3.926032418467635 type value nauc_precision_at_1000_max 19.08045437036499 type value nauc_precision_at_1000_std 6.796129044597931 type value nauc_precision_at_100_diff1 9.73958477384916 type value nauc_precision_at_100_max 29.07096859484853 type value nauc_precision_at_100_std 12.96991105140292 type value nauc_precision_at_10_diff1 17.1980255233314 type value nauc_precision_at_10_max 43.52273606745023 type value nauc_precision_at_10_std 10.958034153583304 type value nauc_precision_at_1_diff1 37.27173489316227 type value nauc_precision_at_1_max 40.3355905491979 type value nauc_precision_at_1_std 6.933728079474825 type value nauc_precision_at_20_diff1 15.369446454490415 type value nauc_precision_at_20_max 39.48680497589929 type value nauc_precision_at_20_std 12.929898425260022 type value nauc_precision_at_3_diff1 23.95767667939835 type value nauc_precision_at_3_max 45.09931497087944 type value nauc_precision_at_3_std 8.770453872723321 type value nauc_precision_at_5_diff1 21.065461642416665 type value nauc_precision_at_5_max 44.72202962458711 type value nauc_precision_at_5_std 10.750776410192397 type value nauc_recall_at_1000_diff1 6.039018739578919 type value nauc_recall_at_1000_max 11.436015450640827 type value nauc_recall_at_1000_std 66.07591419148011 type value nauc_recall_at_100_diff1 18.65963295269777 type value nauc_recall_at_100_max 38.12793731004431 type value nauc_recall_at_100_std 22.877750142093177 type value nauc_recall_at_10_diff1 19.342831730746934 type value nauc_recall_at_10_max 31.63232417200137 type value nauc_recall_at_10_std 3.271699563530961 type value nauc_recall_at_1_diff1 37.6388258626095 type value nauc_recall_at_1_max 25.192760889147102 type value nauc_recall_at_1_std 0.84012346712856 type value nauc_recall_at_20_diff1 20.61391084945006 type value nauc_recall_at_20_max 32.5842740511191 type value nauc_recall_at_20_std 9.819500541742485 type value nauc_recall_at_3_diff1 28.687240532045937 type value nauc_recall_at_3_max 31.72988302079546 type value nauc_recall_at_3_std 2.7062500297360295 type value nauc_recall_at_5_diff1 25.62354289856022 type value nauc_recall_at_5_max 32.13732981730723 type value nauc_recall_at_5_std 4.661623958812741 type value ndcg_at_1 27.236 type value ndcg_at_10 34.795 type value ndcg_at_100 44.352000000000004 type value ndcg_at_1000 46.98 type value ndcg_at_20 38.537 type value ndcg_at_3 29.448 type value ndcg_at_5 30.996000000000002 type value precision_at_1 27.236 type value precision_at_10 10.427 type value precision_at_100 1.8769999999999998 type value precision_at_1000 0.22300000000000003 type value precision_at_20 6.589 type value precision_at_3 21.584 type value precision_at_5 16.555 type value recall_at_1 13.818 type value recall_at_10 44.826 type value recall_at_100 82.047 type value recall_at_1000 99.286 type value recall_at_20 56.615 type value recall_at_3 28.509 type value recall_at_5 35.472 type Retrieval dataset metrics task config name revision split type fra-eng MTEB XPQARetrieval (fra-eng) c99d599f0a6ab9b85b065da6f9d94f9cf731679f test jinaai/xpqa type value main_score 55.165 type value map_at_1 30.070999999999998 type value map_at_10 48.391 type value map_at_100 50.077000000000005 type value map_at_1000 50.175000000000004 type value map_at_20 49.425999999999995 type value map_at_3 43.108999999999995 type value map_at_5 46.331 type value mrr_at_1 47.79706275033378 type value mrr_at_10 57.112499205289545 type value mrr_at_100 57.77653857554601 type value mrr_at_1000 57.810309267669155 type value mrr_at_20 57.50639791688323 type value mrr_at_3 55.028927458833955 type value mrr_at_5 56.19715175789934 type value nauc_map_at_1000_diff1 48.30744877651571 type value nauc_map_at_1000_max 41.304811375830106 type value nauc_map_at_1000_std -3.319640191562977 type value nauc_map_at_100_diff1 48.24792131158136 type value nauc_map_at_100_max 41.289809346155096 type value nauc_map_at_100_std -3.3322490840754044 type value nauc_map_at_10_diff1 48.51735869545944 type value nauc_map_at_10_max 41.39091467858207 type value nauc_map_at_10_std -3.816834529081366 type value nauc_map_at_1_diff1 55.20976873300869 type value nauc_map_at_1_max 29.31564406699001 type value nauc_map_at_1_std -3.531295202744916 type value nauc_map_at_20_diff1 48.15994357210226 type value nauc_map_at_20_max 41.27059573974859 type value nauc_map_at_20_std -3.553567850461392 type value nauc_map_at_3_diff1 49.07638331745524 type value nauc_map_at_3_max 37.4344180429373 type value nauc_map_at_3_std -4.793107974869855 type value nauc_map_at_5_diff1 48.610911544450566 type value nauc_map_at_5_max 40.36936409939194 type value nauc_map_at_5_std -4.494930285823858 type value nauc_mrr_at_1000_diff1 54.361185624681966 type value nauc_mrr_at_1000_max 44.38223734909631 type value nauc_mrr_at_1000_std -4.6407251183091045 type value nauc_mrr_at_100_diff1 54.3534593950135 type value nauc_mrr_at_100_max 44.378173894610114 type value nauc_mrr_at_100_std -4.625111682775984 type value nauc_mrr_at_10_diff1 54.24312662266002 type value nauc_mrr_at_10_max 44.30746970923868 type value nauc_mrr_at_10_std -4.965217414238369 type value nauc_mrr_at_1_diff1 58.26954724773496 type value nauc_mrr_at_1_max 45.03422518009353 type value nauc_mrr_at_1_std -4.069334933239831 type value nauc_mrr_at_20_diff1 54.25997769977666 type value nauc_mrr_at_20_max 44.38402056799441 type value nauc_mrr_at_20_std -4.671395366726689 type value nauc_mrr_at_3_diff1 54.499361492963985 type value nauc_mrr_at_3_max 43.81936137776164 type value nauc_mrr_at_3_std -5.644776625702544 type value nauc_mrr_at_5_diff1 54.44957576550037 type value nauc_mrr_at_5_max 43.983826735470124 type value nauc_mrr_at_5_std -5.796801921865972 type value nauc_ndcg_at_1000_diff1 49.15241156513385 type value nauc_ndcg_at_1000_max 42.45980120922761 type value nauc_ndcg_at_1000_std -2.3353260193872605 type value nauc_ndcg_at_100_diff1 48.24122686676774 type value nauc_ndcg_at_100_max 42.27679493782058 type value nauc_ndcg_at_100_std -1.5752369584570114 type value nauc_ndcg_at_10_diff1 48.5509813605824 type value nauc_ndcg_at_10_max 42.59298249833255 type value nauc_ndcg_at_10_std -3.672669315491546 type value nauc_ndcg_at_1_diff1 58.26954724773496 type value nauc_ndcg_at_1_max 45.03422518009353 type value nauc_ndcg_at_1_std -4.069334933239831 type value nauc_ndcg_at_20_diff1 47.729261088005316 type value nauc_ndcg_at_20_max 42.49497033902468 type value nauc_ndcg_at_20_std -2.6719433358977773 type value nauc_ndcg_at_3_diff1 48.68223689824344 type value nauc_ndcg_at_3_max 40.9157048148036 type value nauc_ndcg_at_3_std -5.637336437839516 type value nauc_ndcg_at_5_diff1 48.69726991107552 type value nauc_ndcg_at_5_max 41.152294520697076 type value nauc_ndcg_at_5_std -5.48123275220102 type value nauc_precision_at_1000_diff1 -10.425039324403782 type value nauc_precision_at_1000_max 7.051352071885475 type value nauc_precision_at_1000_std 4.456043136940008 type value nauc_precision_at_100_diff1 -6.528489272274514 type value nauc_precision_at_100_max 12.611149343017736 type value nauc_precision_at_100_std 5.918229501417929 type value nauc_precision_at_10_diff1 9.37469315859335 type value nauc_precision_at_10_max 29.792160957981938 type value nauc_precision_at_10_std 0.2316309488416353 type value nauc_precision_at_1_diff1 58.26954724773496 type value nauc_precision_at_1_max 45.03422518009353 type value nauc_precision_at_1_std -4.069334933239831 type value nauc_precision_at_20_diff1 2.981751622851337 type value nauc_precision_at_20_max 23.312084195651227 type value nauc_precision_at_20_std 2.560521133286893 type value nauc_precision_at_3_diff1 20.831474725533468 type value nauc_precision_at_3_max 34.732843194059996 type value nauc_precision_at_3_std -3.379064346220114 type value nauc_precision_at_5_diff1 14.628778037588857 type value nauc_precision_at_5_max 33.5567398421705 type value nauc_precision_at_5_std -2.4525869923256236 type value nauc_recall_at_1000_diff1 24.629562614981076 type value nauc_recall_at_1000_max 37.74776159843809 type value nauc_recall_at_1000_std 45.84365921167674 type value nauc_recall_at_100_diff1 28.656294603430176 type value nauc_recall_at_100_max 34.99333512037935 type value nauc_recall_at_100_std 18.07167333451945 type value nauc_recall_at_10_diff1 39.579271628779686 type value nauc_recall_at_10_max 39.65055294313406 type value nauc_recall_at_10_std -1.4953189564586904 type value nauc_recall_at_1_diff1 55.20976873300869 type value nauc_recall_at_1_max 29.31564406699001 type value nauc_recall_at_1_std -3.531295202744916 type value nauc_recall_at_20_diff1 35.59952531108398 type value nauc_recall_at_20_max 39.735665662589234 type value nauc_recall_at_20_std 2.746812413081314 type value nauc_recall_at_3_diff1 42.180790443876234 type value nauc_recall_at_3_max 33.23529070499019 type value nauc_recall_at_3_std -7.102867270573987 type value nauc_recall_at_5_diff1 41.34875509720362 type value nauc_recall_at_5_max 36.67737500141328 type value nauc_recall_at_5_std -7.16711230678949 type value ndcg_at_1 47.797 type value ndcg_at_10 55.165 type value ndcg_at_100 61.072 type value ndcg_at_1000 62.766999999999996 type value ndcg_at_20 57.603 type value ndcg_at_3 50.134 type value ndcg_at_5 51.711 type value precision_at_1 47.797 type value precision_at_10 13.150999999999998 type value precision_at_100 1.8370000000000002 type value precision_at_1000 0.20600000000000002 type value precision_at_20 7.517 type value precision_at_3 30.975 type value precision_at_5 22.27 type value recall_at_1 30.070999999999998 type value recall_at_10 65.352 type value recall_at_100 88.31099999999999 type value recall_at_1000 99.417 type value recall_at_20 72.65 type value recall_at_3 49.891000000000005 type value recall_at_5 56.949000000000005 type Retrieval dataset metrics task config name revision split type default MTEB AllegroReviews b89853e6de927b0e3bfa8ecc0e56fe4e02ceafc6 test PL-MTEB/allegro-reviews type value accuracy 53.48906560636182 type value f1 41.948000361532074 type value f1_weighted 50.64284561538599 type value main_score 53.48906560636182 type Classification dataset metrics task config name revision split type default MTEB ArguAna-PL 63fc86750af76253e8c760fc9e534bbf24d260a2 test clarin-knext/arguana-pl type value main_score 49.913000000000004 type value map_at_1 24.04 type value map_at_10 40.493 type value map_at_100 41.447 type value map_at_1000 41.454 type value map_at_20 41.197 type value map_at_3 35.099999999999994 type value map_at_5 38.196999999999996 type value mrr_at_1 24.537695590327168 type value mrr_at_10 40.67259929102034 type value mrr_at_100 41.639618460436125 type value mrr_at_1000 41.64596845247576 type value mrr_at_20 41.38915253517258 type value mrr_at_3 35.27738264580362 type value mrr_at_5 38.40327169274532 type value nauc_map_at_1000_diff1 7.431509810863732 type value nauc_map_at_1000_max -2.981272220393634 type value nauc_map_at_1000_std -7.60710973485905 type value nauc_map_at_100_diff1 7.436737619273204 type value nauc_map_at_100_max -2.967184788185936 type value nauc_map_at_100_std -7.597426337410871 type value nauc_map_at_10_diff1 7.255093659685807 type value nauc_map_at_10_max -2.9042962900147544 type value nauc_map_at_10_std -7.934694729089717 type value nauc_map_at_1_diff1 12.509203312194646 type value nauc_map_at_1_max -5.881727148045224 type value nauc_map_at_1_std -7.791332615643759 type value nauc_map_at_20_diff1 7.327100008464186 type value nauc_map_at_20_max -2.837417061935196 type value nauc_map_at_20_std -7.727026459254324 type value nauc_map_at_3_diff1 6.852993254257847 type value nauc_map_at_3_max -4.051470844228069 type value nauc_map_at_3_std -7.896963683580916 type value nauc_map_at_5_diff1 6.528299731268904 type value nauc_map_at_5_max -3.6970340215693476 type value nauc_map_at_5_std -7.655276417735266 type value nauc_mrr_at_1000_diff1 5.711449969160694 type value nauc_mrr_at_1000_max -3.4753506470039266 type value nauc_mrr_at_1000_std -7.794020380041222 type value nauc_mrr_at_100_diff1 5.717019799542202 type value nauc_mrr_at_100_max -3.461221495753972 type value nauc_mrr_at_100_std -7.784340755281538 type value nauc_mrr_at_10_diff1 5.509993731954919 type value nauc_mrr_at_10_max -3.4562614853854345 type value nauc_mrr_at_10_std -8.172557318463994 type value nauc_mrr_at_1_diff1 10.815838583441858 type value nauc_mrr_at_1_max -5.323382194534891 type value nauc_mrr_at_1_std -8.038288156705363 type value nauc_mrr_at_20_diff1 5.622966175346149 type value nauc_mrr_at_20_max -3.3271519171448602 type value nauc_mrr_at_20_std -7.911979321248223 type value nauc_mrr_at_3_diff1 5.1203118177676945 type value nauc_mrr_at_3_max -4.663436282182911 type value nauc_mrr_at_3_std -8.16687342201878 type value nauc_mrr_at_5_diff1 4.899936200607895 type value nauc_mrr_at_5_max -4.238888324916206 type value nauc_mrr_at_5_std -7.911378372003927 type value nauc_ndcg_at_1000_diff1 7.208621858675132 type value nauc_ndcg_at_1000_max -1.9047927444267347 type value nauc_ndcg_at_1000_std -6.986137159109878 type value nauc_ndcg_at_100_diff1 7.409545817332008 type value nauc_ndcg_at_100_max -1.4631671846013694 type value nauc_ndcg_at_100_std -6.630280309037233 type value nauc_ndcg_at_10_diff1 6.4667756391170395 type value nauc_ndcg_at_10_max -0.6950268010456382 type value nauc_ndcg_at_10_std -8.022144927522392 type value nauc_ndcg_at_1_diff1 12.509203312194646 type value nauc_ndcg_at_1_max -5.881727148045224 type value nauc_ndcg_at_1_std -7.791332615643759 type value nauc_ndcg_at_20_diff1 6.726279074146785 type value nauc_ndcg_at_20_max -0.3861052348420354 type value nauc_ndcg_at_20_std -7.221277273790139 type value nauc_ndcg_at_3_diff1 5.5538863803913365 type value nauc_ndcg_at_3_max -3.5651217527245946 type value nauc_ndcg_at_3_std -7.826880086024049 type value nauc_ndcg_at_5_diff1 4.878905871379252 type value nauc_ndcg_at_5_max -2.821048486985759 type value nauc_ndcg_at_5_std -7.31598311150453 type value nauc_precision_at_1000_diff1 31.595672412803232 type value nauc_precision_at_1000_max 42.56487657246246 type value nauc_precision_at_1000_std 76.77064740096077 type value nauc_precision_at_100_diff1 37.959767569852325 type value nauc_precision_at_100_max 61.03819238774345 type value nauc_precision_at_100_std 57.75475522584779 type value nauc_precision_at_10_diff1 3.679895666980749 type value nauc_precision_at_10_max 11.38829056417457 type value nauc_precision_at_10_std -8.650914185729293 type value nauc_precision_at_1_diff1 12.509203312194646 type value nauc_precision_at_1_max -5.881727148045224 type value nauc_precision_at_1_std -7.791332615643759 type value nauc_precision_at_20_diff1 4.065515107956777 type value nauc_precision_at_20_max 23.888067135216097 type value nauc_precision_at_20_std -1.4622436922054596 type value nauc_precision_at_3_diff1 2.1003082872796663 type value nauc_precision_at_3_max -2.24675019839533 type value nauc_precision_at_3_std -7.604178336955303 type value nauc_precision_at_5_diff1 -0.246824792648523 type value nauc_precision_at_5_max 0.0642032358424201 type value nauc_precision_at_5_std -6.0892549043276745 type value nauc_recall_at_1000_diff1 31.59567241280578 type value nauc_recall_at_1000_max 42.564876572459895 type value nauc_recall_at_1000_std 76.7706474009625 type value nauc_recall_at_100_diff1 37.95976756985206 type value nauc_recall_at_100_max 61.03819238774383 type value nauc_recall_at_100_std 57.75475522584684 type value nauc_recall_at_10_diff1 3.679895666980674 type value nauc_recall_at_10_max 11.388290564174538 type value nauc_recall_at_10_std -8.650914185729265 type value nauc_recall_at_1_diff1 12.509203312194646 type value nauc_recall_at_1_max -5.881727148045224 type value nauc_recall_at_1_std -7.791332615643759 type value nauc_recall_at_20_diff1 4.065515107957231 type value nauc_recall_at_20_max 23.888067135216005 type value nauc_recall_at_20_std -1.462243692205622 type value nauc_recall_at_3_diff1 2.100308287279676 type value nauc_recall_at_3_max -2.2467501983953024 type value nauc_recall_at_3_std -7.604178336955286 type value nauc_recall_at_5_diff1 -0.24682479264852286 type value nauc_recall_at_5_max 0.06420323584243659 type value nauc_recall_at_5_std -6.089254904327643 type value ndcg_at_1 24.04 type value ndcg_at_10 49.913000000000004 type value ndcg_at_100 54.057 type value ndcg_at_1000 54.213 type value ndcg_at_20 52.42400000000001 type value ndcg_at_3 38.842999999999996 type value ndcg_at_5 44.416 type value precision_at_1 24.04 type value precision_at_10 8.009 type value precision_at_100 0.984 type value precision_at_1000 0.1 type value precision_at_20 4.495 type value precision_at_3 16.572 type value precision_at_5 12.645999999999999 type value recall_at_1 24.04 type value recall_at_10 80.085 type value recall_at_100 98.36399999999999 type value recall_at_1000 99.57300000000001 type value recall_at_20 89.9 type value recall_at_3 49.716 type value recall_at_5 63.229 type Retrieval dataset metrics task config name revision split type default MTEB CBD 36ddb419bcffe6a5374c3891957912892916f28d test PL-MTEB/cbd type value accuracy 64.91 type value ap 20.253009474993238 type value ap_weighted 20.253009474993238 type value f1 54.83698737303514 type value f1_weighted 69.53194816160229 type value main_score 64.91 type Classification dataset metrics task config name revision split type default MTEB CDSC-E 0a3d4aa409b22f80eb22cbf59b492637637b536d test PL-MTEB/cdsce-pairclassification type value cosine_accuracy 87.5 type value cosine_accuracy_threshold 97.39001989364624 type value cosine_ap 71.63899566137869 type value cosine_f1 64.39024390243902 type value cosine_f1_threshold 94.18535828590393 type value cosine_precision 60.0 type value cosine_recall 69.47368421052632 type value dot_accuracy 87.5 type value dot_accuracy_threshold 97.39001989364624 type value dot_ap 71.63899566137869 type value dot_f1 64.39024390243902 type value dot_f1_threshold 94.18535232543945 type value dot_precision 60.0 type value dot_recall 69.47368421052632 type value euclidean_accuracy 87.5 type value euclidean_accuracy_threshold 22.847232222557068 type value euclidean_ap 71.63899566137869 type value euclidean_f1 64.39024390243902 type value euclidean_f1_threshold 34.101736545562744 type value euclidean_precision 60.0 type value euclidean_recall 69.47368421052632 type value main_score 71.83631821171632 type value manhattan_accuracy 87.6 type value manhattan_accuracy_threshold 499.97105598449707 type value manhattan_ap 71.83631821171632 type value manhattan_f1 64.5631067961165 type value manhattan_f1_threshold 809.0234756469727 type value manhattan_precision 59.909909909909906 type value manhattan_recall 70.0 type value max_ap 71.83631821171632 type value max_f1 64.5631067961165 type value max_precision 60.0 type value max_recall 70.0 type value similarity_accuracy 87.5 type value similarity_accuracy_threshold 97.39001989364624 type value similarity_ap 71.63899566137869 type value similarity_f1 64.39024390243902 type value similarity_f1_threshold 94.18535828590393 type value similarity_precision 60.0 type value similarity_recall 69.47368421052632 type PairClassification dataset metrics task config name revision split type default MTEB CDSC-R 1cd6abbb00df7d14be3dbd76a7dcc64b3a79a7cd test PL-MTEB/cdscr-sts type value cosine_pearson 89.9839992597087 type value cosine_spearman 90.27044716786627 type value euclidean_pearson 87.74719535276023 type value euclidean_spearman 90.2703874383013 type value main_score 90.27044716786627 type value manhattan_pearson 87.81149530960033 type value manhattan_spearman 90.37098083828207 type value pearson 89.9839992597087 type value spearman 90.27044716786627 type STS dataset metrics task config name revision split type default MTEB DBPedia-PL 76afe41d9af165cc40999fcaa92312b8b012064a test clarin-knext/dbpedia-pl type value main_score 29.225 type value map_at_1 5.92 type value map_at_10 13.052 type value map_at_100 18.054000000000002 type value map_at_1000 19.378999999999998 type value map_at_20 14.921000000000001 type value map_at_3 9.517000000000001 type value map_at_5 11.122 type value mrr_at_1 45.0 type value mrr_at_10 57.3967261904762 type value mrr_at_100 57.83804567388388 type value mrr_at_1000 57.86075000832548 type value mrr_at_20 57.66969785675282 type value mrr_at_3 55.16666666666667 type value mrr_at_5 56.64166666666669 type value nauc_map_at_1000_diff1 29.411798531506246 type value nauc_map_at_1000_max 20.900134633305655 type value nauc_map_at_1000_std 31.404039472246353 type value nauc_map_at_100_diff1 30.843903551109808 type value nauc_map_at_100_max 17.39151067247246 type value nauc_map_at_100_std 27.44650726590824 type value nauc_map_at_10_diff1 37.979613569219495 type value nauc_map_at_10_max 9.222700346624988 type value nauc_map_at_10_std 12.007799385555293 type value nauc_map_at_1_diff1 53.50284116730185 type value nauc_map_at_1_max 1.370522275254312 type value nauc_map_at_1_std -0.30640006292692257 type value nauc_map_at_20_diff1 35.67559578714465 type value nauc_map_at_20_max 12.765002402346221 type value nauc_map_at_20_std 17.73265858605054 type value nauc_map_at_3_diff1 45.619789003130585 type value nauc_map_at_3_max 1.045838638341231 type value nauc_map_at_3_std 2.319308580529236 type value nauc_map_at_5_diff1 42.08058689946505 type value nauc_map_at_5_max 5.337616164644746 type value nauc_map_at_5_std 4.73118790791731 type value nauc_mrr_at_1000_diff1 34.33930133013396 type value nauc_mrr_at_1000_max 29.38799773918778 type value nauc_mrr_at_1000_std 32.26009048699902 type value nauc_mrr_at_100_diff1 34.3197444457885 type value nauc_mrr_at_100_max 29.413059576309497 type value nauc_mrr_at_100_std 32.26908951100588 type value nauc_mrr_at_10_diff1 34.30610810384026 type value nauc_mrr_at_10_max 29.25358347303212 type value nauc_mrr_at_10_std 32.42735770220712 type value nauc_mrr_at_1_diff1 38.47836050546717 type value nauc_mrr_at_1_max 25.549990178746796 type value nauc_mrr_at_1_std 27.017285405617763 type value nauc_mrr_at_20_diff1 34.32685063678914 type value nauc_mrr_at_20_max 29.382152716878547 type value nauc_mrr_at_20_std 32.36225065070027 type value nauc_mrr_at_3_diff1 34.94513788944085 type value nauc_mrr_at_3_max 28.948106098297938 type value nauc_mrr_at_3_std 31.752978523564845 type value nauc_mrr_at_5_diff1 34.22773791436512 type value nauc_mrr_at_5_max 28.645995406061914 type value nauc_mrr_at_5_std 31.947761641656065 type value nauc_ndcg_at_1000_diff1 23.59930215160307 type value nauc_ndcg_at_1000_max 30.004827423326873 type value nauc_ndcg_at_1000_std 45.14606063029462 type value nauc_ndcg_at_100_diff1 27.150265390833766 type value nauc_ndcg_at_100_max 21.542350038665962 type value nauc_ndcg_at_100_std 37.04783459199997 type value nauc_ndcg_at_10_diff1 30.44928623138369 type value nauc_ndcg_at_10_max 21.38523283782705 type value nauc_ndcg_at_10_std 31.948655996496527 type value nauc_ndcg_at_1_diff1 38.141954118151105 type value nauc_ndcg_at_1_max 20.764788523221725 type value nauc_ndcg_at_1_std 24.457971796268065 type value nauc_ndcg_at_20_diff1 31.668458090974728 type value nauc_ndcg_at_20_max 20.1903988669924 type value nauc_ndcg_at_20_std 30.646872442412544 type value nauc_ndcg_at_3_diff1 30.030850630038053 type value nauc_ndcg_at_3_max 19.919461574491066 type value nauc_ndcg_at_3_std 28.065728170179188 type value nauc_ndcg_at_5_diff1 30.06324115773368 type value nauc_ndcg_at_5_max 21.013491210996943 type value nauc_ndcg_at_5_std 29.390767365137947 type value nauc_precision_at_1000_diff1 -15.2968288893292 type value nauc_precision_at_1000_max 48.371418703337305 type value nauc_precision_at_1000_std 33.90852748893144 type value nauc_precision_at_100_diff1 -7.607176962046647 type value nauc_precision_at_100_max 35.35122884806948 type value nauc_precision_at_100_std 46.4742326977524 type value nauc_precision_at_10_diff1 0.0234083902358811 type value nauc_precision_at_10_max 34.310462135642645 type value nauc_precision_at_10_std 46.22745495492598 type value nauc_precision_at_1_diff1 38.47836050546717 type value nauc_precision_at_1_max 25.549990178746796 type value nauc_precision_at_1_std 27.017285405617763 type value nauc_precision_at_20_diff1 -0.7281234339501458 type value nauc_precision_at_20_max 34.879992298927796 type value nauc_precision_at_20_std 46.6455237720046 type value nauc_precision_at_3_diff1 12.557632325001943 type value nauc_precision_at_3_max 27.472641291674343 type value nauc_precision_at_3_std 32.76253410590738 type value nauc_precision_at_5_diff1 5.72403051661784 type value nauc_precision_at_5_max 31.623557984213747 type value nauc_precision_at_5_std 37.60956680129879 type value nauc_recall_at_1000_diff1 5.745409852861974 type value nauc_recall_at_1000_max 27.497512598172698 type value nauc_recall_at_1000_std 48.07303762126119 type value nauc_recall_at_100_diff1 17.211282922855617 type value nauc_recall_at_100_max 17.98582110327383 type value nauc_recall_at_100_std 34.86455715009784 type value nauc_recall_at_10_diff1 28.755279638184874 type value nauc_recall_at_10_max 8.106029595934537 type value nauc_recall_at_10_std 12.493783688335569 type value nauc_recall_at_1_diff1 53.50284116730185 type value nauc_recall_at_1_max 1.370522275254312 type value nauc_recall_at_1_std -0.30640006292692257 type value nauc_recall_at_20_diff1 27.994527440411993 type value nauc_recall_at_20_max 12.916323071056604 type value nauc_recall_at_20_std 17.70928825635808 type value nauc_recall_at_3_diff1 39.80550258552395 type value nauc_recall_at_3_max -0.8593780074939045 type value nauc_recall_at_3_std 2.086691158003704 type value nauc_recall_at_5_diff1 34.29080510342918 type value nauc_recall_at_5_max 2.8885937240283113 type value nauc_recall_at_5_std 2.6609799835271852 type value ndcg_at_1 35.875 type value ndcg_at_10 29.225 type value ndcg_at_100 33.554 type value ndcg_at_1000 40.908 type value ndcg_at_20 28.910000000000004 type value ndcg_at_3 32.405 type value ndcg_at_5 30.408 type value precision_at_1 45.0 type value precision_at_10 23.599999999999998 type value precision_at_100 7.68 type value precision_at_1000 1.804 type value precision_at_20 17.5 type value precision_at_3 36.167 type value precision_at_5 30.15 type value recall_at_1 5.92 type value recall_at_10 18.658 type value recall_at_100 40.144999999999996 type value recall_at_1000 63.914 type value recall_at_20 23.91 type value recall_at_3 11.334 type value recall_at_5 14.251 type Retrieval dataset metrics task config name revision split type default MTEB 8TagsClustering 78b962b130c6690659c65abf67bf1c2f030606b6 test PL-MTEB/8tags-clustering type value main_score 37.57372573379629 type value v_measure 37.57372573379629 type value v_measure_std 1.576502898019969 type Clustering dataset metrics task config name revision split type default MTEB FiQA-PL 2e535829717f8bf9dc829b7f911cc5bbd4e6608e test clarin-knext/fiqa-pl type value main_score 25.322 type value map_at_1 12.084 type value map_at_10 19.402 type value map_at_100 20.766000000000002 type value map_at_1000 20.958 type value map_at_20 20.085 type value map_at_3 16.794 type value map_at_5 18.242 type value mrr_at_1 23.30246913580247 type value mrr_at_10 31.084594846168915 type value mrr_at_100 32.081458268143486 type value mrr_at_1000 32.15082259510916 type value mrr_at_20 31.641799089124518 type value mrr_at_3 28.703703703703713 type value mrr_at_5 30.12345679012346 type value nauc_map_at_1000_diff1 33.497391865616606 type value nauc_map_at_1000_max 15.431683878656488 type value nauc_map_at_1000_std 10.827813213986468 type value nauc_map_at_100_diff1 33.534068616502886 type value nauc_map_at_100_max 15.291439989133599 type value nauc_map_at_100_std 10.715061061847777 type value nauc_map_at_10_diff1 33.49437614167937 type value nauc_map_at_10_max 14.377484560226964 type value nauc_map_at_10_std 9.487834206589557 type value nauc_map_at_1_diff1 39.87810373637443 type value nauc_map_at_1_max 10.730137705508765 type value nauc_map_at_1_std 3.2660873686456195 type value nauc_map_at_20_diff1 33.37736866727796 type value nauc_map_at_20_max 14.70143784805556 type value nauc_map_at_20_std 9.989663285421791 type value nauc_map_at_3_diff1 34.368864609204216 type value nauc_map_at_3_max 12.768667645519768 type value nauc_map_at_3_std 7.982752811874638 type value nauc_map_at_5_diff1 33.58267051366728 type value nauc_map_at_5_max 13.529005222918848 type value nauc_map_at_5_std 8.565140707894367 type value nauc_mrr_at_1000_diff1 34.518749214862446 type value nauc_mrr_at_1000_max 20.004412541379317 type value nauc_mrr_at_1000_std 10.794450592562008 type value nauc_mrr_at_100_diff1 34.502828469831684 type value nauc_mrr_at_100_max 20.016402128122674 type value nauc_mrr_at_100_std 10.770953740589398 type value nauc_mrr_at_10_diff1 34.464123530074744 type value nauc_mrr_at_10_max 19.812317084561315 type value nauc_mrr_at_10_std 10.660604975440622 type value nauc_mrr_at_1_diff1 39.735267543303344 type value nauc_mrr_at_1_max 20.218792748481526 type value nauc_mrr_at_1_std 7.574870456628672 type value nauc_mrr_at_20_diff1 34.4112636812203 type value nauc_mrr_at_20_max 19.736403323847995 type value nauc_mrr_at_20_std 10.58825811173397 type value nauc_mrr_at_3_diff1 34.322321922524765 type value nauc_mrr_at_3_max 19.48120229919887 type value nauc_mrr_at_3_std 10.241852033769396 type value nauc_mrr_at_5_diff1 34.41273362560696 type value nauc_mrr_at_5_max 19.80166599189298 type value nauc_mrr_at_5_std 10.535257678547225 type value nauc_ndcg_at_1000_diff1 31.756209625205372 type value nauc_ndcg_at_1000_max 19.79815198505404 type value nauc_ndcg_at_1000_std 15.747292429924494 type value nauc_ndcg_at_100_diff1 32.24612802150064 type value nauc_ndcg_at_100_max 18.490724459073633 type value nauc_ndcg_at_100_std 14.606523975785374 type value nauc_ndcg_at_10_diff1 32.17599943968043 type value nauc_ndcg_at_10_max 15.73203247263979 type value nauc_ndcg_at_10_std 11.361059016427816 type value nauc_ndcg_at_1_diff1 39.735267543303344 type value nauc_ndcg_at_1_max 20.218792748481526 type value nauc_ndcg_at_1_std 7.574870456628672 type value nauc_ndcg_at_20_diff1 31.750276068192886 type value nauc_ndcg_at_20_max 15.761403266813346 type value nauc_ndcg_at_20_std 11.939341736048261 type value nauc_ndcg_at_3_diff1 32.60001850916417 type value nauc_ndcg_at_3_max 16.484580482661286 type value nauc_ndcg_at_3_std 9.93945065513519 type value nauc_ndcg_at_5_diff1 32.44524427279313 type value nauc_ndcg_at_5_max 15.875506598237141 type value nauc_ndcg_at_5_std 9.982281820511833 type value nauc_precision_at_1000_diff1 5.371199115978502 type value nauc_precision_at_1000_max 32.2390464051828 type value nauc_precision_at_1000_std 14.878904307648414 type value nauc_precision_at_100_diff1 16.16681952079101 type value nauc_precision_at_100_max 31.799356005933838 type value nauc_precision_at_100_std 19.248994737500986 type value nauc_precision_at_10_diff1 22.009585966198923 type value nauc_precision_at_10_max 25.75349877480564 type value nauc_precision_at_10_std 16.27236030310856 type value nauc_precision_at_1_diff1 39.735267543303344 type value nauc_precision_at_1_max 20.218792748481526 type value nauc_precision_at_1_std 7.574870456628672 type value nauc_precision_at_20_diff1 18.58140182399686 type value nauc_precision_at_20_max 25.678514022441874 type value nauc_precision_at_20_std 16.797936080303757 type value nauc_precision_at_3_diff1 26.928025721272824 type value nauc_precision_at_3_max 20.657641661666794 type value nauc_precision_at_3_std 13.0985390930848 type value nauc_precision_at_5_diff1 23.36859898010871 type value nauc_precision_at_5_max 22.374908445175237 type value nauc_precision_at_5_std 14.246505892972294 type value nauc_recall_at_1000_diff1 11.980972712740272 type value nauc_recall_at_1000_max 19.76758314007667 type value nauc_recall_at_1000_std 37.01896226544845 type value nauc_recall_at_100_diff1 21.23333081030157 type value nauc_recall_at_100_max 17.273702477754753 type value nauc_recall_at_100_std 22.66184024937999 type value nauc_recall_at_10_diff1 24.654784002876422 type value nauc_recall_at_10_max 11.299238954418193 type value nauc_recall_at_10_std 12.933536657323804 type value nauc_recall_at_1_diff1 39.87810373637443 type value nauc_recall_at_1_max 10.730137705508765 type value nauc_recall_at_1_std 3.2660873686456195 type value nauc_recall_at_20_diff1 22.912968265183142 type value nauc_recall_at_20_max 10.463163094071744 type value nauc_recall_at_20_std 13.342666469120315 type value nauc_recall_at_3_diff1 26.200195626449702 type value nauc_recall_at_3_max 10.661728055293116 type value nauc_recall_at_3_std 10.101882781882052 type value nauc_recall_at_5_diff1 25.286289446845807 type value nauc_recall_at_5_max 11.353540373539142 type value nauc_recall_at_5_std 10.67026258089847 type value ndcg_at_1 23.302 type value ndcg_at_10 25.322 type value ndcg_at_100 31.452 type value ndcg_at_1000 35.378 type value ndcg_at_20 27.392 type value ndcg_at_3 22.238 type value ndcg_at_5 23.436 type value precision_at_1 23.302 type value precision_at_10 7.037 type value precision_at_100 1.321 type value precision_at_1000 0.2 type value precision_at_20 4.344 type value precision_at_3 14.557999999999998 type value precision_at_5 10.988000000000001 type value recall_at_1 12.084 type value recall_at_10 31.011 type value recall_at_100 54.782 type value recall_at_1000 78.828 type value recall_at_20 37.573 type value recall_at_3 20.918999999999997 type value recall_at_5 25.434 type Retrieval dataset metrics task config name revision split type default MTEB HotpotQA-PL a0bd479ac97b4ccb5bd6ce320c415d0bb4beb907 test clarin-knext/hotpotqa-pl type value main_score 61.76199999999999 type value map_at_1 36.462 type value map_at_10 52.595000000000006 type value map_at_100 53.486 type value map_at_1000 53.561 type value map_at_20 53.116 type value map_at_3 49.55 type value map_at_5 51.468 type value mrr_at_1 72.92370020256584 type value mrr_at_10 79.14170498269061 type value mrr_at_100 79.39082829565201 type value mrr_at_1000 79.4039312237504 type value mrr_at_20 79.30320990617905 type value mrr_at_3 78.06887238352448 type value mrr_at_5 78.74746792707597 type value nauc_map_at_1000_diff1 26.629197478945656 type value nauc_map_at_1000_max 20.417296536263652 type value nauc_map_at_1000_std 7.824861166949661 type value nauc_map_at_100_diff1 26.597747680876964 type value nauc_map_at_100_max 20.394321293004854 type value nauc_map_at_100_std 7.812277969136019 type value nauc_map_at_10_diff1 26.733323682484784 type value nauc_map_at_10_max 20.271344228458663 type value nauc_map_at_10_std 7.0935616016511815 type value nauc_map_at_1_diff1 73.40480136620272 type value nauc_map_at_1_max 38.86815860879837 type value nauc_map_at_1_std 4.8325955891477275 type value nauc_map_at_20_diff1 26.568842010897114 type value nauc_map_at_20_max 20.275169904863905 type value nauc_map_at_20_std 7.56661656432979 type value nauc_map_at_3_diff1 28.824845889064793 type value nauc_map_at_3_max 20.76852907202902 type value nauc_map_at_3_std 5.754512537392399 type value nauc_map_at_5_diff1 27.454615905979974 type value nauc_map_at_5_max 20.352277144385937 type value nauc_map_at_5_std 6.601409288581079 type value nauc_mrr_at_1000_diff1 72.29337975556386 type value nauc_mrr_at_1000_max 41.162812968303555 type value nauc_mrr_at_1000_std 7.658983139015768 type value nauc_mrr_at_100_diff1 72.28963649528013 type value nauc_mrr_at_100_max 41.16405855619647 type value nauc_mrr_at_100_std 7.671105812656405 type value nauc_mrr_at_10_diff1 72.20735283859506 type value nauc_mrr_at_10_max 41.22707207638071 type value nauc_mrr_at_10_std 7.642216005282447 type value nauc_mrr_at_1_diff1 73.40480136620272 type value nauc_mrr_at_1_max 38.86815860879837 type value nauc_mrr_at_1_std 4.8325955891477275 type value nauc_mrr_at_20_diff1 72.28084176981353 type value nauc_mrr_at_20_max 41.19699794135133 type value nauc_mrr_at_20_std 7.673602725654943 type value nauc_mrr_at_3_diff1 72.2517312298736 type value nauc_mrr_at_3_max 41.23050336709122 type value nauc_mrr_at_3_std 7.055398076214827 type value nauc_mrr_at_5_diff1 72.3010580466702 type value nauc_mrr_at_5_max 41.16023128418148 type value nauc_mrr_at_5_std 7.224799100313062 type value nauc_ndcg_at_1000_diff1 31.836096618552684 type value nauc_ndcg_at_1000_max 24.19594101782851 type value nauc_ndcg_at_1000_std 11.27051039772318 type value nauc_ndcg_at_100_diff1 31.010910429281985 type value nauc_ndcg_at_100_max 23.73763527936943 type value nauc_ndcg_at_100_std 11.202567249866915 type value nauc_ndcg_at_10_diff1 31.630736903110733 type value nauc_ndcg_at_10_max 23.29057670190408 type value nauc_ndcg_at_10_std 8.622063436605352 type value nauc_ndcg_at_1_diff1 73.40480136620272 type value nauc_ndcg_at_1_max 38.86815860879837 type value nauc_ndcg_at_1_std 4.8325955891477275 type value nauc_ndcg_at_20_diff1 31.022867077795073 type value nauc_ndcg_at_20_max 23.20240329652894 type value nauc_ndcg_at_20_std 9.910412291823127 type value nauc_ndcg_at_3_diff1 35.496569057786346 type value nauc_ndcg_at_3_max 24.448277354535833 type value nauc_ndcg_at_3_std 6.498237519761217 type value nauc_ndcg_at_5_diff1 33.251227793460906 type value nauc_ndcg_at_5_max 23.605853646520984 type value nauc_ndcg_at_5_std 7.54284385208763 type value nauc_precision_at_1000_diff1 -0.47079501803456375 type value nauc_precision_at_1000_max 15.089814566667142 type value nauc_precision_at_1000_std 27.847788246114057 type value nauc_precision_at_100_diff1 3.0595485970514704 type value nauc_precision_at_100_max 14.360431203666717 type value nauc_precision_at_100_std 22.31753410548815 type value nauc_precision_at_10_diff1 11.454235819834814 type value nauc_precision_at_10_max 14.979788854311145 type value nauc_precision_at_10_std 11.290542607411098 type value nauc_precision_at_1_diff1 73.40480136620272 type value nauc_precision_at_1_max 38.86815860879837 type value nauc_precision_at_1_std 4.8325955891477275 type value nauc_precision_at_20_diff1 7.60972218209098 type value nauc_precision_at_20_max 13.692113405742418 type value nauc_precision_at_20_std 15.359273788872974 type value nauc_precision_at_3_diff1 22.002230799209492 type value nauc_precision_at_3_max 19.075064977055266 type value nauc_precision_at_3_std 7.1760372858256956 type value nauc_precision_at_5_diff1 16.565606958337607 type value nauc_precision_at_5_max 16.550935196750206 type value nauc_precision_at_5_std 8.807234374696868 type value nauc_recall_at_1000_diff1 -0.47079501803429247 type value nauc_recall_at_1000_max 15.089814566667334 type value nauc_recall_at_1000_std 27.847788246114025 type value nauc_recall_at_100_diff1 3.0595485970514558 type value nauc_recall_at_100_max 14.360431203666705 type value nauc_recall_at_100_std 22.317534105488054 type value nauc_recall_at_10_diff1 11.4542358198349 type value nauc_recall_at_10_max 14.979788854311154 type value nauc_recall_at_10_std 11.290542607411085 type value nauc_recall_at_1_diff1 73.40480136620272 type value nauc_recall_at_1_max 38.86815860879837 type value nauc_recall_at_1_std 4.8325955891477275 type value nauc_recall_at_20_diff1 7.609722182091017 type value nauc_recall_at_20_max 13.692113405742424 type value nauc_recall_at_20_std 15.35927378887301 type value nauc_recall_at_3_diff1 22.002230799209435 type value nauc_recall_at_3_max 19.07506497705519 type value nauc_recall_at_3_std 7.176037285825619 type value nauc_recall_at_5_diff1 16.56560695833764 type value nauc_recall_at_5_max 16.55093519675023 type value nauc_recall_at_5_std 8.807234374696902 type value ndcg_at_1 72.924 type value ndcg_at_10 61.76199999999999 type value ndcg_at_100 64.943 type value ndcg_at_1000 66.42 type value ndcg_at_20 63.105 type value ndcg_at_3 57.318000000000005 type value ndcg_at_5 59.80799999999999 type value precision_at_1 72.924 type value precision_at_10 12.723999999999998 type value precision_at_100 1.521 type value precision_at_1000 0.172 type value precision_at_20 6.795 type value precision_at_3 35.863 type value precision_at_5 23.487 type value recall_at_1 36.462 type value recall_at_10 63.619 type value recall_at_100 76.036 type value recall_at_1000 85.8 type value recall_at_20 67.95400000000001 type value recall_at_3 53.795 type value recall_at_5 58.717 type Retrieval dataset metrics task config name revision split type default MTEB MSMARCO-PL 8634c07806d5cce3a6138e260e59b81760a0a640 test clarin-knext/msmarco-pl type value main_score 45.132 type value map_at_1 1.667 type value map_at_10 8.405999999999999 type value map_at_100 20.796 type value map_at_1000 25.679999999999996 type value map_at_20 11.882 type value map_at_3 3.4000000000000004 type value map_at_5 5.289 type value mrr_at_1 62.7906976744186 type value mrr_at_10 71.9767441860465 type value mrr_at_100 72.19001178866145 type value mrr_at_1000 72.21077590826278 type value mrr_at_20 71.9767441860465 type value mrr_at_3 69.76744186046511 type value mrr_at_5 71.9767441860465 type value nauc_map_at_1000_diff1 13.121496890926018 type value nauc_map_at_1000_max 64.4620914971356 type value nauc_map_at_1000_std 70.89107882842627 type value nauc_map_at_100_diff1 6.569373263154751 type value nauc_map_at_100_max 54.52329917268778 type value nauc_map_at_100_std 57.970012281008195 type value nauc_map_at_10_diff1 12.479881525075633 type value nauc_map_at_10_max 16.416934605814358 type value nauc_map_at_10_std 16.562025084061755 type value nauc_map_at_1_diff1 -13.480148625088354 type value nauc_map_at_1_max -12.48386553446901 type value nauc_map_at_1_std -19.47568765990734 type value nauc_map_at_20_diff1 8.75113737642458 type value nauc_map_at_20_max 28.316394733873455 type value nauc_map_at_20_std 28.706433416288757 type value nauc_map_at_3_diff1 0.4892858373106769 type value nauc_map_at_3_max 4.82429174133813 type value nauc_map_at_3_std 2.685691736161667 type value nauc_map_at_5_diff1 7.407280581282287 type value nauc_map_at_5_max 7.810182361989069 type value nauc_map_at_5_std 7.1694430987177915 type value nauc_mrr_at_1000_diff1 -1.3143171207174462 type value nauc_mrr_at_1000_max 55.56132775818817 type value nauc_mrr_at_1000_std 44.747614607383106 type value nauc_mrr_at_100_diff1 -1.224506180649995 type value nauc_mrr_at_100_max 55.600720798015224 type value nauc_mrr_at_100_std 44.73970951740156 type value nauc_mrr_at_10_diff1 -1.404072265069855 type value nauc_mrr_at_10_max 55.81202913496246 type value nauc_mrr_at_10_std 45.1755213724528 type value nauc_mrr_at_1_diff1 -3.3932017924925764 type value nauc_mrr_at_1_max 45.85906083891651 type value nauc_mrr_at_1_std 36.94174294169342 type value nauc_mrr_at_20_diff1 -1.404072265069855 type value nauc_mrr_at_20_max 55.81202913496246 type value nauc_mrr_at_20_std 45.1755213724528 type value nauc_mrr_at_3_diff1 -1.9535315867645546 type value nauc_mrr_at_3_max 54.66533478368106 type value nauc_mrr_at_3_std 42.93031026511843 type value nauc_mrr_at_5_diff1 -1.404072265069855 type value nauc_mrr_at_5_max 55.81202913496246 type value nauc_mrr_at_5_std 45.1755213724528 type value nauc_ndcg_at_1000_diff1 15.612187648926648 type value nauc_ndcg_at_1000_max 66.0369696987196 type value nauc_ndcg_at_1000_std 69.96669745374349 type value nauc_ndcg_at_100_diff1 8.757636842486582 type value nauc_ndcg_at_100_max 60.74693277069104 type value nauc_ndcg_at_100_std 63.76108092965522 type value nauc_ndcg_at_10_diff1 6.45234697262411 type value nauc_ndcg_at_10_max 47.130858592103536 type value nauc_ndcg_at_10_std 46.654922458779126 type value nauc_ndcg_at_1_diff1 -4.400276896768569 type value nauc_ndcg_at_1_max 24.736725318748277 type value nauc_ndcg_at_1_std 15.100951232927404 type value nauc_ndcg_at_20_diff1 -0.44419635404462504 type value nauc_ndcg_at_20_max 53.81470890104093 type value nauc_ndcg_at_20_std 54.65514527813791 type value nauc_ndcg_at_3_diff1 4.176276992379476 type value nauc_ndcg_at_3_max 33.4079755228582 type value nauc_ndcg_at_3_std 26.097236468435497 type value nauc_ndcg_at_5_diff1 9.966039505450683 type value nauc_ndcg_at_5_max 40.118178652342394 type value nauc_ndcg_at_5_std 34.33405125137147 type value nauc_precision_at_1000_diff1 13.757669487153102 type value nauc_precision_at_1000_max 52.007228955531794 type value nauc_precision_at_1000_std 62.70603005119199 type value nauc_precision_at_100_diff1 7.1595084301066105 type value nauc_precision_at_100_max 57.56055309573276 type value nauc_precision_at_100_std 69.09674838687823 type value nauc_precision_at_10_diff1 10.548904389246808 type value nauc_precision_at_10_max 58.361747853932435 type value nauc_precision_at_10_std 62.35890309913381 type value nauc_precision_at_1_diff1 -3.3932017924925764 type value nauc_precision_at_1_max 45.85906083891651 type value nauc_precision_at_1_std 36.94174294169342 type value nauc_precision_at_20_diff1 0.5486557649755647 type value nauc_precision_at_20_max 55.8966200841496 type value nauc_precision_at_20_std 64.46833667077514 type value nauc_precision_at_3_diff1 3.74969726265482 type value nauc_precision_at_3_max 50.98538299147468 type value nauc_precision_at_3_std 47.52256580019106 type value nauc_precision_at_5_diff1 14.409304075805396 type value nauc_precision_at_5_max 52.63426384539844 type value nauc_precision_at_5_std 48.72540538657435 type value nauc_recall_at_1000_diff1 14.810856570503505 type value nauc_recall_at_1000_max 56.70402594077228 type value nauc_recall_at_1000_std 62.44988045776601 type value nauc_recall_at_100_diff1 -0.547033022823402 type value nauc_recall_at_100_max 37.5943435400723 type value nauc_recall_at_100_std 42.055737611040904 type value nauc_recall_at_10_diff1 5.6072575274918695 type value nauc_recall_at_10_max 6.244507044627988 type value nauc_recall_at_10_std 5.1959433044082575 type value nauc_recall_at_1_diff1 -13.480148625088354 type value nauc_recall_at_1_max -12.48386553446901 type value nauc_recall_at_1_std -19.47568765990734 type value nauc_recall_at_20_diff1 1.5008424440815344 type value nauc_recall_at_20_max 16.711622731636748 type value nauc_recall_at_20_std 16.46978349884905 type value nauc_recall_at_3_diff1 -2.3329900069251996 type value nauc_recall_at_3_max 2.511711071593615 type value nauc_recall_at_3_std -0.5855889251226093 type value nauc_recall_at_5_diff1 4.1075104414046315 type value nauc_recall_at_5_max 0.34189966462509463 type value nauc_recall_at_5_std -1.89085195502975 type value ndcg_at_1 50.0 type value ndcg_at_10 45.132 type value ndcg_at_100 41.504999999999995 type value ndcg_at_1000 49.738 type value ndcg_at_20 42.569 type value ndcg_at_3 45.423 type value ndcg_at_5 45.611000000000004 type value precision_at_1 62.791 type value precision_at_10 54.419 type value precision_at_100 25.047000000000004 type value precision_at_1000 5.002 type value precision_at_20 46.394999999999996 type value precision_at_3 57.364000000000004 type value precision_at_5 57.208999999999996 type value recall_at_1 1.667 type value recall_at_10 10.933 type value recall_at_100 35.169 type value recall_at_1000 59.955999999999996 type value recall_at_20 16.399 type value recall_at_3 3.7379999999999995 type value recall_at_5 6.365 type Retrieval dataset metrics task config name revision split type pl MTEB MassiveIntentClassification (pl) 4672e20407010da34463acc759c162ca9734bca6 test mteb/amazon_massive_intent type value accuracy 62.55548083389375 type value f1 55.243883281423955 type value f1_weighted 61.53554902108963 type value main_score 62.55548083389375 type Classification dataset metrics task config name revision split type pl MTEB MassiveScenarioClassification (pl) fad2c6e8459f9e1c45d9315f4953d921437d70f8 test mteb/amazon_massive_scenario type value accuracy 71.7518493611298 type value f1 69.39084021404145 type value f1_weighted 71.48397679382578 type value main_score 71.7518493611298 type Classification dataset metrics task config name revision split type default MTEB NFCorpus-PL 9a6f9567fda928260afed2de480d79c98bf0bec0 test clarin-knext/nfcorpus-pl type value main_score 27.359 type value map_at_1 4.013 type value map_at_10 9.243 type value map_at_100 11.417 type value map_at_1000 12.465 type value map_at_20 10.241999999999999 type value map_at_3 6.6739999999999995 type value map_at_5 7.720000000000001 type value mrr_at_1 36.84210526315789 type value mrr_at_10 45.80704211509165 type value mrr_at_100 46.43056530919217 type value mrr_at_1000 46.481813685972384 type value mrr_at_20 46.2328011230761 type value mrr_at_3 43.653250773993804 type value mrr_at_5 44.75232198142416 type value nauc_map_at_1000_diff1 24.84177430292285 type value nauc_map_at_1000_max 17.115036682746375 type value nauc_map_at_1000_std 24.075727964418853 type value nauc_map_at_100_diff1 25.813465171019708 type value nauc_map_at_100_max 15.890774948775189 type value nauc_map_at_100_std 20.733065453457606 type value nauc_map_at_10_diff1 29.488943622716107 type value nauc_map_at_10_max 9.776720754233569 type value nauc_map_at_10_std 10.581345052422016 type value nauc_map_at_1_diff1 48.57974934948135 type value nauc_map_at_1_max 1.149527115405564 type value nauc_map_at_1_std -2.0301983395175363 type value nauc_map_at_20_diff1 27.744545091489826 type value nauc_map_at_20_max 12.800210322701194 type value nauc_map_at_20_std 15.036851255880851 type value nauc_map_at_3_diff1 37.25540055051418 type value nauc_map_at_3_max 4.906473702901897 type value nauc_map_at_3_std 1.462933406016024 type value nauc_map_at_5_diff1 33.75262117705747 type value nauc_map_at_5_max 5.349094540200769 type value nauc_map_at_5_std 4.009473353212513 type value nauc_mrr_at_1000_diff1 25.923316236906224 type value nauc_mrr_at_1000_max 30.218473131172814 type value nauc_mrr_at_1000_std 34.32841034971355 type value nauc_mrr_at_100_diff1 25.89160877435761 type value nauc_mrr_at_100_max 30.26076316909358 type value nauc_mrr_at_100_std 34.38168790885202 type value nauc_mrr_at_10_diff1 25.94165965662626 type value nauc_mrr_at_10_max 29.92861838955619 type value nauc_mrr_at_10_std 34.217857324602384 type value nauc_mrr_at_1_diff1 27.77544038178182 type value nauc_mrr_at_1_max 23.544571519690063 type value nauc_mrr_at_1_std 29.133288904288985 type value nauc_mrr_at_20_diff1 25.817823276199377 type value nauc_mrr_at_20_max 30.212951519162534 type value nauc_mrr_at_20_std 34.38656845672502 type value nauc_mrr_at_3_diff1 27.253167791083772 type value nauc_mrr_at_3_max 28.668229911423044 type value nauc_mrr_at_3_std 32.24039598508148 type value nauc_mrr_at_5_diff1 26.50152942042588 type value nauc_mrr_at_5_max 29.014104429398657 type value nauc_mrr_at_5_std 33.10408829199384 type value nauc_ndcg_at_1000_diff1 21.670441606508682 type value nauc_ndcg_at_1000_max 35.085480170350294 type value nauc_ndcg_at_1000_std 40.26959838435534 type value nauc_ndcg_at_100_diff1 20.56655267151386 type value nauc_ndcg_at_100_max 29.059496472106172 type value nauc_ndcg_at_100_std 36.20604882231693 type value nauc_ndcg_at_10_diff1 19.327892822047392 type value nauc_ndcg_at_10_max 22.970443207173847 type value nauc_ndcg_at_10_std 33.63485024562264 type value nauc_ndcg_at_1_diff1 29.440869586898806 type value nauc_ndcg_at_1_max 21.1892146993199 type value nauc_ndcg_at_1_std 27.715145294772626 type value nauc_ndcg_at_20_diff1 19.84119342340242 type value nauc_ndcg_at_20_max 24.648907071153918 type value nauc_ndcg_at_20_std 34.21144991558109 type value nauc_ndcg_at_3_diff1 22.475236266303952 type value nauc_ndcg_at_3_max 22.5673625414089 type value nauc_ndcg_at_3_std 30.40344427150939 type value nauc_ndcg_at_5_diff1 20.435706146454795 type value nauc_ndcg_at_5_max 20.807509478884405 type value nauc_ndcg_at_5_std 30.50756403953348 type value nauc_precision_at_1000_diff1 -7.734779276193169 type value nauc_precision_at_1000_max 10.369447288094234 type value nauc_precision_at_1000_std 38.88122374339474 type value nauc_precision_at_100_diff1 -5.148267935551239 type value nauc_precision_at_100_max 22.682811622480507 type value nauc_precision_at_100_std 52.14414978661011 type value nauc_precision_at_10_diff1 4.2440553409575115 type value nauc_precision_at_10_max 24.922198902459577 type value nauc_precision_at_10_std 44.24729160099345 type value nauc_precision_at_1_diff1 28.683873179972423 type value nauc_precision_at_1_max 24.333474443231477 type value nauc_precision_at_1_std 29.657103597064992 type value nauc_precision_at_20_diff1 0.981459375147628 type value nauc_precision_at_20_max 26.656822900511944 type value nauc_precision_at_20_std 47.61829905274704 type value nauc_precision_at_3_diff1 14.009226282963393 type value nauc_precision_at_3_max 25.206963221334643 type value nauc_precision_at_3_std 34.640163356829575 type value nauc_precision_at_5_diff1 9.732199396026699 type value nauc_precision_at_5_max 21.620896160839308 type value nauc_precision_at_5_std 36.54829562203162 type value nauc_recall_at_1000_diff1 13.592706145413594 type value nauc_recall_at_1000_max 26.905710458923515 type value nauc_recall_at_1000_std 27.77232599212786 type value nauc_recall_at_100_diff1 11.474980161550619 type value nauc_recall_at_100_max 24.6542606788053 type value nauc_recall_at_100_std 26.088933416325894 type value nauc_recall_at_10_diff1 20.86627786542471 type value nauc_recall_at_10_max 12.310575849201342 type value nauc_recall_at_10_std 8.93720284107538 type value nauc_recall_at_1_diff1 48.57974934948135 type value nauc_recall_at_1_max 1.149527115405564 type value nauc_recall_at_1_std -2.0301983395175363 type value nauc_recall_at_20_diff1 17.03977114136929 type value nauc_recall_at_20_max 15.132361504438405 type value nauc_recall_at_20_std 14.39504435329145 type value nauc_recall_at_3_diff1 33.90735954186142 type value nauc_recall_at_3_max 7.589690453066397 type value nauc_recall_at_3_std 0.8609172933612455 type value nauc_recall_at_5_diff1 27.37452904528661 type value nauc_recall_at_5_max 6.950034812753282 type value nauc_recall_at_5_std 2.9248007586594396 type value ndcg_at_1 35.294 type value ndcg_at_10 27.359 type value ndcg_at_100 24.285999999999998 type value ndcg_at_1000 32.438 type value ndcg_at_20 25.418000000000003 type value ndcg_at_3 31.328 type value ndcg_at_5 29.269000000000002 type value precision_at_1 36.533 type value precision_at_10 20.681 type value precision_at_100 6.087 type value precision_at_1000 1.7469999999999999 type value precision_at_20 15.325 type value precision_at_3 29.309 type value precision_at_5 25.201 type value recall_at_1 4.013 type value recall_at_10 13.153 type value recall_at_100 24.549000000000003 type value recall_at_1000 53.908 type value recall_at_20 16.453 type value recall_at_3 7.832999999999999 type value recall_at_5 9.693999999999999 type Retrieval dataset metrics task config name revision split type default MTEB NQ-PL f171245712cf85dd4700b06bef18001578d0ca8d test clarin-knext/nq-pl type value main_score 30.842000000000002 type value map_at_1 15.584999999999999 type value map_at_10 25.141999999999996 type value map_at_100 26.387 type value map_at_1000 26.458 type value map_at_20 25.897 type value map_at_3 21.792 type value map_at_5 23.605 type value mrr_at_1 17.526071842410197 type value mrr_at_10 27.034281943754777 type value mrr_at_100 28.093499231975112 type value mrr_at_1000 28.151579697181628 type value mrr_at_20 27.685578601768064 type value mrr_at_3 23.966782541521876 type value mrr_at_5 25.63538045577454 type value nauc_map_at_1000_diff1 25.629659206470034 type value nauc_map_at_1000_max 19.50903133109958 type value nauc_map_at_1000_std 11.369355803540456 type value nauc_map_at_100_diff1 25.63185640379452 type value nauc_map_at_100_max 19.49043016244933 type value nauc_map_at_100_std 11.349471698782217 type value nauc_map_at_10_diff1 25.801905100212085 type value nauc_map_at_10_max 18.71914313595772 type value nauc_map_at_10_std 10.101933080218412 type value nauc_map_at_1_diff1 27.69756013829008 type value nauc_map_at_1_max 13.265356278967614 type value nauc_map_at_1_std 4.845453511488002 type value nauc_map_at_20_diff1 25.57617091165384 type value nauc_map_at_20_max 19.22087134146287 type value nauc_map_at_20_std 10.863338999338074 type value nauc_map_at_3_diff1 26.04936647826419 type value nauc_map_at_3_max 17.00014461889098 type value nauc_map_at_3_std 8.345803797704802 type value nauc_map_at_5_diff1 25.926914766086163 type value nauc_map_at_5_max 17.909768342318312 type value nauc_map_at_5_std 8.99533665314055 type value nauc_mrr_at_1000_diff1 24.821439280682775 type value nauc_mrr_at_1000_max 20.48215524313607 type value nauc_mrr_at_1000_std 13.302755245100787 type value nauc_mrr_at_100_diff1 24.822888515699727 type value nauc_mrr_at_100_max 20.476125364875305 type value nauc_mrr_at_100_std 13.303370196580808 type value nauc_mrr_at_10_diff1 24.827095834283377 type value nauc_mrr_at_10_max 19.906455259365014 type value nauc_mrr_at_10_std 12.461215626420783 type value nauc_mrr_at_1_diff1 27.354076617153282 type value nauc_mrr_at_1_max 15.421589080989397 type value nauc_mrr_at_1_std 7.854191402321044 type value nauc_mrr_at_20_diff1 24.707829956282353 type value nauc_mrr_at_20_max 20.343614549048684 type value nauc_mrr_at_20_std 12.991368337778994 type value nauc_mrr_at_3_diff1 25.001495195422212 type value nauc_mrr_at_3_max 18.670877184315987 type value nauc_mrr_at_3_std 11.073823459359353 type value nauc_mrr_at_5_diff1 25.09633485104506 type value nauc_mrr_at_5_max 19.289598809877393 type value nauc_mrr_at_5_std 11.447861090124427 type value nauc_ndcg_at_1000_diff1 24.454331896090252 type value nauc_ndcg_at_1000_max 24.54817880813177 type value nauc_ndcg_at_1000_std 18.291577235898664 type value nauc_ndcg_at_100_diff1 24.4900499476292 type value nauc_ndcg_at_100_max 24.3113863055596 type value nauc_ndcg_at_100_std 18.283249505464127 type value nauc_ndcg_at_10_diff1 24.75304628631047 type value nauc_ndcg_at_10_max 21.346414904765112 type value nauc_ndcg_at_10_std 13.144087870627114 type value nauc_ndcg_at_1_diff1 27.354076617153282 type value nauc_ndcg_at_1_max 15.421589080989397 type value nauc_ndcg_at_1_std 7.854191402321044 type value nauc_ndcg_at_20_diff1 24.054443970465634 type value nauc_ndcg_at_20_max 23.02090178343728 type value nauc_ndcg_at_20_std 15.466706732549639 type value nauc_ndcg_at_3_diff1 25.21593203645425 type value nauc_ndcg_at_3_max 18.366389791319857 type value nauc_ndcg_at_3_std 9.886764558221312 type value nauc_ndcg_at_5_diff1 25.18968308632415 type value nauc_ndcg_at_5_max 19.714457143715883 type value nauc_ndcg_at_5_std 10.810267333820615 type value nauc_precision_at_1000_diff1 5.311743560049695 type value nauc_precision_at_1000_max 31.8449636551786 type value nauc_precision_at_1000_std 38.560980646256645 type value nauc_precision_at_100_diff1 11.642708984639716 type value nauc_precision_at_100_max 33.08348545702312 type value nauc_precision_at_100_std 38.84569611188958 type value nauc_precision_at_10_diff1 19.39529546701617 type value nauc_precision_at_10_max 27.35329522618733 type value nauc_precision_at_10_std 21.657982938733863 type value nauc_precision_at_1_diff1 27.354076617153282 type value nauc_precision_at_1_max 15.421589080989397 type value nauc_precision_at_1_std 7.854191402321044 type value nauc_precision_at_20_diff1 15.315200424520157 type value nauc_precision_at_20_max 30.813032263448335 type value nauc_precision_at_20_std 28.51929835139947 type value nauc_precision_at_3_diff1 23.171414749401624 type value nauc_precision_at_3_max 22.230781193639906 type value nauc_precision_at_3_std 14.39995607518812 type value nauc_precision_at_5_diff1 22.12050049652593 type value nauc_precision_at_5_max 24.47739013891615 type value nauc_precision_at_5_std 15.911936861665232 type value nauc_recall_at_1000_diff1 18.49721947186244 type value nauc_recall_at_1000_max 59.77562391547361 type value nauc_recall_at_1000_std 67.25992226904116 type value nauc_recall_at_100_diff1 21.08120571727416 type value nauc_recall_at_100_max 41.81711687017934 type value nauc_recall_at_100_std 45.46881224307712 type value nauc_recall_at_10_diff1 22.267969061265276 type value nauc_recall_at_10_max 26.20350836241132 type value nauc_recall_at_10_std 18.312586912516927 type value nauc_recall_at_1_diff1 27.69756013829008 type value nauc_recall_at_1_max 13.265356278967614 type value nauc_recall_at_1_std 4.845453511488002 type value nauc_recall_at_20_diff1 19.7184358966775 type value nauc_recall_at_20_max 32.18279692099271 type value nauc_recall_at_20_std 26.185137240814377 type value nauc_recall_at_3_diff1 23.501740451271914 type value nauc_recall_at_3_max 19.91360673787573 type value nauc_recall_at_3_std 11.210024942573977 type value nauc_recall_at_5_diff1 23.437183434421655 type value nauc_recall_at_5_max 22.272023416475623 type value nauc_recall_at_5_std 12.814496156956142 type value ndcg_at_1 17.526 type value ndcg_at_10 30.842000000000002 type value ndcg_at_100 36.629 type value ndcg_at_1000 38.495000000000005 type value ndcg_at_20 33.382 type value ndcg_at_3 24.252000000000002 type value ndcg_at_5 27.339000000000002 type value precision_at_1 17.526 type value precision_at_10 5.548 type value precision_at_100 0.88 type value precision_at_1000 0.106 type value precision_at_20 3.3649999999999998 type value precision_at_3 11.25 type value precision_at_5 8.517 type value recall_at_1 15.584999999999999 type value recall_at_10 46.521 type value recall_at_100 72.571 type value recall_at_1000 86.86500000000001 type value recall_at_20 56.004 type value recall_at_3 29.195999999999998 type value recall_at_5 36.324 type Retrieval dataset metrics task config name revision split type default MTEB PAC fc69d1c153a8ccdcf1eef52f4e2a27f88782f543 test laugustyniak/abusive-clauses-pl type value accuracy 64.80162177816392 type value ap 74.10348604798286 type value ap_weighted 74.10348604798286 type value f1 61.280331645723685 type value f1_weighted 65.03859489177282 type value main_score 64.80162177816392 type Classification dataset metrics task config name revision split type default MTEB PSC d05a294af9e1d3ff2bfb6b714e08a24a6cabc669 test PL-MTEB/psc-pairclassification type value cosine_accuracy 97.77365491651206 type value cosine_accuracy_threshold 81.08445405960083 type value cosine_ap 99.43195082030653 type value cosine_f1 96.40718562874251 type value cosine_f1_threshold 81.08445405960083 type value cosine_precision 94.70588235294117 type value cosine_recall 98.17073170731707 type value dot_accuracy 97.77365491651206 type value dot_accuracy_threshold 81.08445405960083 type value dot_ap 99.43195082030653 type value dot_f1 96.40718562874251 type value dot_f1_threshold 81.08445405960083 type value dot_precision 94.70588235294117 type value dot_recall 98.17073170731707 type value euclidean_accuracy 97.77365491651206 type value euclidean_accuracy_threshold 61.50695085525513 type value euclidean_ap 99.43195082030653 type value euclidean_f1 96.40718562874251 type value euclidean_f1_threshold 61.50695085525513 type value euclidean_precision 94.70588235294117 type value euclidean_recall 98.17073170731707 type value main_score 99.46339853695966 type value manhattan_accuracy 98.05194805194806 type value manhattan_accuracy_threshold 1428.3578872680664 type value manhattan_ap 99.46339853695966 type value manhattan_f1 96.83257918552036 type value manhattan_f1_threshold 1428.3578872680664 type value manhattan_precision 95.82089552238806 type value manhattan_recall 97.86585365853658 type value max_ap 99.46339853695966 type value max_f1 96.83257918552036 type value max_precision 95.82089552238806 type value max_recall 98.17073170731707 type value similarity_accuracy 97.77365491651206 type value similarity_accuracy_threshold 81.08445405960083 type value similarity_ap 99.43195082030653 type value similarity_f1 96.40718562874251 type value similarity_f1_threshold 81.08445405960083 type value similarity_precision 94.70588235294117 type value similarity_recall 98.17073170731707 type PairClassification dataset metrics task config name revision split type default MTEB PolEmo2.0-IN d90724373c70959f17d2331ad51fb60c71176b03 test PL-MTEB/polemo2_in type value accuracy 80.54016620498614 type value f1 74.07868803329357 type value f1_weighted 78.52375884318697 type value main_score 80.54016620498614 type Classification dataset metrics task config name revision split type default MTEB PolEmo2.0-OUT 6a21ab8716e255ab1867265f8b396105e8aa63d4 test PL-MTEB/polemo2_out type value accuracy 61.37651821862349 type value f1 46.60510896853889 type value f1_weighted 61.3956699958363 type value main_score 61.37651821862349 type Classification dataset metrics task config name revision split type default MTEB PPC 2c7d2df57801a591f6b1e3aaf042e7a04ec7d9f2 test PL-MTEB/ppc-pairclassification type value cosine_accuracy 76.0 type value cosine_accuracy_threshold 93.07277202606201 type value cosine_ap 87.43755817552731 type value cosine_f1 80.46989720998532 type value cosine_f1_threshold 90.98483324050903 type value cosine_precision 72.29551451187335 type value cosine_recall 90.72847682119205 type value dot_accuracy 76.0 type value dot_accuracy_threshold 93.07277798652649 type value dot_ap 87.43751021710085 type value dot_f1 80.46989720998532 type value dot_f1_threshold 90.98482728004456 type value dot_precision 72.29551451187335 type value dot_recall 90.72847682119205 type value euclidean_accuracy 76.0 type value euclidean_accuracy_threshold 37.221553921699524 type value euclidean_ap 87.43751021710085 type value euclidean_f1 80.46989720998532 type value euclidean_f1_threshold 42.46214032173157 type value euclidean_precision 72.29551451187335 type value euclidean_recall 90.72847682119205 type value main_score 87.43755817552731 type value manhattan_accuracy 75.2 type value manhattan_accuracy_threshold 858.4394454956055 type value manhattan_ap 87.28751334847506 type value manhattan_f1 80.47162859248341 type value manhattan_f1_threshold 981.0188293457031 type value manhattan_precision 72.50996015936255 type value manhattan_recall 90.39735099337747 type value max_ap 87.43755817552731 type value max_f1 80.47162859248341 type value max_precision 72.50996015936255 type value max_recall 90.72847682119205 type value similarity_accuracy 76.0 type value similarity_accuracy_threshold 93.07277202606201 type value similarity_ap 87.43755817552731 type value similarity_f1 80.46989720998532 type value similarity_f1_threshold 90.98483324050903 type value similarity_precision 72.29551451187335 type value similarity_recall 90.72847682119205 type PairClassification dataset metrics task config name revision split type default MTEB Quora-PL 0be27e93455051e531182b85e85e425aba12e9d4 test clarin-knext/quora-pl type value main_score 75.235 type value map_at_1 57.720000000000006 type value map_at_10 70.322 type value map_at_100 71.208 type value map_at_1000 71.247 type value map_at_20 70.889 type value map_at_3 67.278 type value map_at_5 69.07900000000001 type value mrr_at_1 66.44 type value mrr_at_10 74.32428571428532 type value mrr_at_100 74.67001717307676 type value mrr_at_1000 74.68049849872023 type value mrr_at_20 74.55920910032467 type value mrr_at_3 72.6349999999996 type value mrr_at_5 73.67099999999938 type value nauc_map_at_1000_diff1 69.03523613954961 type value nauc_map_at_1000_max 30.29022964222993 type value nauc_map_at_1000_std -13.13676129820498 type value nauc_map_at_100_diff1 69.03918889242972 type value nauc_map_at_100_max 30.28851815152789 type value nauc_map_at_100_std -13.173343854637487 type value nauc_map_at_10_diff1 69.11834037559699 type value nauc_map_at_10_max 29.609089948792388 type value nauc_map_at_10_std -14.511647137697395 type value nauc_map_at_1_diff1 72.50653845898617 type value nauc_map_at_1_max 22.521228683262873 type value nauc_map_at_1_std -17.72541519468729 type value nauc_map_at_20_diff1 69.0572096712263 type value nauc_map_at_20_max 30.09049337817234 type value nauc_map_at_20_std -13.69213787699562 type value nauc_map_at_3_diff1 69.4118549460786 type value nauc_map_at_3_max 27.31606724944123 type value nauc_map_at_3_std -16.430296769671298 type value nauc_map_at_5_diff1 69.18608931793607 type value nauc_map_at_5_max 28.681802217476093 type value nauc_map_at_5_std -15.492619374306827 type value nauc_mrr_at_1000_diff1 70.27871731978331 type value nauc_mrr_at_1000_max 33.89585229097829 type value nauc_mrr_at_1000_std -9.231498078778678 type value nauc_mrr_at_100_diff1 70.27656223213475 type value nauc_mrr_at_100_max 33.90583650980198 type value nauc_mrr_at_100_std -9.213247629622375 type value nauc_mrr_at_10_diff1 70.1800255282438 type value nauc_mrr_at_10_max 33.975132933927085 type value nauc_mrr_at_10_std -9.344439026014577 type value nauc_mrr_at_1_diff1 72.72425945481199 type value nauc_mrr_at_1_max 31.239650246117385 type value nauc_mrr_at_1_std -11.607242701686696 type value nauc_mrr_at_20_diff1 70.24166041655792 type value nauc_mrr_at_20_max 33.9613048334359 type value nauc_mrr_at_20_std -9.219736983314839 type value nauc_mrr_at_3_diff1 70.06664104900666 type value nauc_mrr_at_3_max 33.5732140539362 type value nauc_mrr_at_3_std -9.778577982149953 type value nauc_mrr_at_5_diff1 70.14739007028493 type value nauc_mrr_at_5_max 33.796518466305834 type value nauc_mrr_at_5_std -9.649151783176043 type value nauc_ndcg_at_1000_diff1 68.62634218438664 type value nauc_ndcg_at_1000_max 33.057143795018696 type value nauc_ndcg_at_1000_std -9.563352961803663 type value nauc_ndcg_at_100_diff1 68.58213175533443 type value nauc_ndcg_at_100_max 33.35336572393414 type value nauc_ndcg_at_100_std -9.127811506992467 type value nauc_ndcg_at_10_diff1 68.26726256015203 type value nauc_ndcg_at_10_max 32.33115112923283 type value nauc_ndcg_at_10_std -11.874276014971688 type value nauc_ndcg_at_1_diff1 72.66000012395291 type value nauc_ndcg_at_1_max 31.283711202542207 type value nauc_ndcg_at_1_std -11.501503096057867 type value nauc_ndcg_at_20_diff1 68.39658663907474 type value nauc_ndcg_at_20_max 33.08529095010713 type value nauc_ndcg_at_20_std -10.437492609480433 type value nauc_ndcg_at_3_diff1 68.05324210316826 type value nauc_ndcg_at_3_max 30.30824001099573 type value nauc_ndcg_at_3_std -13.044199992428771 type value nauc_ndcg_at_5_diff1 68.10994364753626 type value nauc_ndcg_at_5_max 31.182072802471055 type value nauc_ndcg_at_5_std -12.836057047748234 type value nauc_precision_at_1000_diff1 -32.848796455727836 type value nauc_precision_at_1000_max 6.715546095139156 type value nauc_precision_at_1000_std 32.9655373056535 type value nauc_precision_at_100_diff1 -28.794521134307093 type value nauc_precision_at_100_max 11.155432738297682 type value nauc_precision_at_100_std 33.30986182557851 type value nauc_precision_at_10_diff1 -10.613535245108128 type value nauc_precision_at_10_max 19.057316698279582 type value nauc_precision_at_10_std 19.87457963908978 type value nauc_precision_at_1_diff1 72.66000012395291 type value nauc_precision_at_1_max 31.283711202542207 type value nauc_precision_at_1_std -11.501503096057867 type value nauc_precision_at_20_diff1 -19.6984185276961 type value nauc_precision_at_20_max 16.497527862287058 type value nauc_precision_at_20_std 26.871607334073012 type value nauc_precision_at_3_diff1 17.130494007304765 type value nauc_precision_at_3_max 23.99199625132106 type value nauc_precision_at_3_std 5.234797091652211 type value nauc_precision_at_5_diff1 3.0202641879085697 type value nauc_precision_at_5_max 22.31257369308076 type value nauc_precision_at_5_std 12.502866671883032 type value nauc_recall_at_1000_diff1 49.899967761974196 type value nauc_recall_at_1000_max 54.39990257883846 type value nauc_recall_at_1000_std 42.663306287015196 type value nauc_recall_at_100_diff1 57.87887190551234 type value nauc_recall_at_100_max 48.03395851487758 type value nauc_recall_at_100_std 25.008694604591312 type value nauc_recall_at_10_diff1 60.99359933290845 type value nauc_recall_at_10_max 34.817508290483154 type value nauc_recall_at_10_std -10.355946195658207 type value nauc_recall_at_1_diff1 72.50653845898617 type value nauc_recall_at_1_max 22.521228683262873 type value nauc_recall_at_1_std -17.72541519468729 type value nauc_recall_at_20_diff1 59.63721580389802 type value nauc_recall_at_20_max 39.78324293003396 type value nauc_recall_at_20_std -0.7738431870195353 type value nauc_recall_at_3_diff1 64.28146361759069 type value nauc_recall_at_3_max 27.55821665783294 type value nauc_recall_at_3_std -16.385154477134336 type value nauc_recall_at_5_diff1 62.687585623754046 type value nauc_recall_at_5_max 30.357420406058328 type value nauc_recall_at_5_std -14.95291415876769 type value ndcg_at_1 66.47 type value ndcg_at_10 75.235 type value ndcg_at_100 77.847 type value ndcg_at_1000 78.396 type value ndcg_at_20 76.539 type value ndcg_at_3 71.219 type value ndcg_at_5 73.235 type value precision_at_1 66.47 type value precision_at_10 11.596 type value precision_at_100 1.424 type value precision_at_1000 0.153 type value precision_at_20 6.331 type value precision_at_3 31.130000000000003 type value precision_at_5 20.735999999999997 type value recall_at_1 57.720000000000006 type value recall_at_10 85.249 type value recall_at_100 95.39699999999999 type value recall_at_1000 98.81 type value recall_at_20 89.739 type value recall_at_3 73.978 type value recall_at_5 79.355 type Retrieval dataset metrics task config name revision split type default MTEB SCIDOCS-PL 45452b03f05560207ef19149545f168e596c9337 test clarin-knext/scidocs-pl type value main_score 15.174000000000001 type value map_at_1 3.6580000000000004 type value map_at_10 8.796 type value map_at_100 10.391 type value map_at_1000 10.646 type value map_at_20 9.592 type value map_at_3 6.489000000000001 type value map_at_5 7.600999999999999 type value mrr_at_1 18.0 type value mrr_at_10 26.845317460317457 type value mrr_at_100 28.04995949015167 type value mrr_at_1000 28.121893269944824 type value mrr_at_20 27.566026091211864 type value mrr_at_3 23.916666666666686 type value mrr_at_5 25.551666666666666 type value nauc_map_at_1000_diff1 17.302827041650488 type value nauc_map_at_1000_max 26.65992706695422 type value nauc_map_at_1000_std 18.96964501922404 type value nauc_map_at_100_diff1 17.21226432890004 type value nauc_map_at_100_max 26.45824637348571 type value nauc_map_at_100_std 18.573352847100065 type value nauc_map_at_10_diff1 17.02056023363081 type value nauc_map_at_10_max 24.48428170985602 type value nauc_map_at_10_std 14.014378375804235 type value nauc_map_at_1_diff1 21.638506619768716 type value nauc_map_at_1_max 19.709230810058283 type value nauc_map_at_1_std 9.042419739024966 type value nauc_map_at_20_diff1 17.067893569553323 type value nauc_map_at_20_max 25.69106547536296 type value nauc_map_at_20_std 16.535327068913993 type value nauc_map_at_3_diff1 18.56349850011108 type value nauc_map_at_3_max 22.127177599224744 type value nauc_map_at_3_std 9.47260767358392 type value nauc_map_at_5_diff1 18.05585009830461 type value nauc_map_at_5_max 23.31477343090323 type value nauc_map_at_5_std 11.257936348356862 type value nauc_mrr_at_1000_diff1 19.71318833342125 type value nauc_mrr_at_1000_max 22.359300102570092 type value nauc_mrr_at_1000_std 13.89561747692388 type value nauc_mrr_at_100_diff1 19.709804653242603 type value nauc_mrr_at_100_max 22.365551370687967 type value nauc_mrr_at_100_std 13.918573803759068 type value nauc_mrr_at_10_diff1 19.74677273038544 type value nauc_mrr_at_10_max 22.348783997030335 type value nauc_mrr_at_10_std 13.606175345418963 type value nauc_mrr_at_1_diff1 21.957688664351128 type value nauc_mrr_at_1_max 19.50356102866365 type value nauc_mrr_at_1_std 9.323755394169037 type value nauc_mrr_at_20_diff1 19.5076818806823 type value nauc_mrr_at_20_max 22.192342439483934 type value nauc_mrr_at_20_std 13.705438410110608 type value nauc_mrr_at_3_diff1 19.784830140193804 type value nauc_mrr_at_3_max 21.606746947165416 type value nauc_mrr_at_3_std 12.289045699872666 type value nauc_mrr_at_5_diff1 20.139962218896674 type value nauc_mrr_at_5_max 22.139813460789266 type value nauc_mrr_at_5_std 13.177813432176084 type value nauc_ndcg_at_1000_diff1 17.78059204124948 type value nauc_ndcg_at_1000_max 29.830544327132436 type value nauc_ndcg_at_1000_std 28.03254237837783 type value nauc_ndcg_at_100_diff1 17.62481104076364 type value nauc_ndcg_at_100_max 28.629131876483665 type value nauc_ndcg_at_100_std 26.019853664301124 type value nauc_ndcg_at_10_diff1 17.25237540570343 type value nauc_ndcg_at_10_max 25.128032787033604 type value nauc_ndcg_at_10_std 16.571629975349868 type value nauc_ndcg_at_1_diff1 21.957688664351128 type value nauc_ndcg_at_1_max 19.50356102866365 type value nauc_ndcg_at_1_std 9.323755394169037 type value nauc_ndcg_at_20_diff1 16.549388210526494 type value nauc_ndcg_at_20_max 26.1871953370256 type value nauc_ndcg_at_20_std 19.971064555030125 type value nauc_ndcg_at_3_diff1 18.707127276019474 type value nauc_ndcg_at_3_max 22.042786711511813 type value nauc_ndcg_at_3_std 11.103829353868623 type value nauc_ndcg_at_5_diff1 18.45321448876598 type value nauc_ndcg_at_5_max 23.475902453066492 type value nauc_ndcg_at_5_std 13.216222368946411 type value nauc_precision_at_1000_diff1 11.843768977161584 type value nauc_precision_at_1000_max 30.300299347010352 type value nauc_precision_at_1000_std 41.123748924498585 type value nauc_precision_at_100_diff1 13.765676375073074 type value nauc_precision_at_100_max 29.769561801824956 type value nauc_precision_at_100_std 37.56343888054612 type value nauc_precision_at_10_diff1 14.123009605345343 type value nauc_precision_at_10_max 26.045793706986558 type value nauc_precision_at_10_std 20.45802977436883 type value nauc_precision_at_1_diff1 21.957688664351128 type value nauc_precision_at_1_max 19.50356102866365 type value nauc_precision_at_1_std 9.323755394169037 type value nauc_precision_at_20_diff1 12.080580953868749 type value nauc_precision_at_20_max 26.741203934729374 type value nauc_precision_at_20_std 26.249289307014976 type value nauc_precision_at_3_diff1 17.390833784290034 type value nauc_precision_at_3_max 22.639415005064585 type value nauc_precision_at_3_std 11.481404394862311 type value nauc_precision_at_5_diff1 17.18007614612505 type value nauc_precision_at_5_max 24.244045184229563 type value nauc_precision_at_5_std 15.180528647694574 type value nauc_recall_at_1000_diff1 11.507406580463488 type value nauc_recall_at_1000_max 30.78976497232251 type value nauc_recall_at_1000_std 41.618419379918855 type value nauc_recall_at_100_diff1 13.408507737517144 type value nauc_recall_at_100_max 29.849796157178197 type value nauc_recall_at_100_std 37.58778281760627 type value nauc_recall_at_10_diff1 13.942112101503866 type value nauc_recall_at_10_max 26.228452951171487 type value nauc_recall_at_10_std 20.14835260352246 type value nauc_recall_at_1_diff1 21.638506619768716 type value nauc_recall_at_1_max 19.709230810058283 type value nauc_recall_at_1_std 9.042419739024966 type value nauc_recall_at_20_diff1 11.905542570350702 type value nauc_recall_at_20_max 26.84107459006622 type value nauc_recall_at_20_std 25.888986621614645 type value nauc_recall_at_3_diff1 17.056201299401692 type value nauc_recall_at_3_max 22.94288018834461 type value nauc_recall_at_3_std 11.337560544201224 type value nauc_recall_at_5_diff1 16.89022137209632 type value nauc_recall_at_5_max 24.564195711081545 type value nauc_recall_at_5_std 14.979769166201622 type value ndcg_at_1 18.0 type value ndcg_at_10 15.174000000000001 type value ndcg_at_100 22.047 type value ndcg_at_1000 27.057 type value ndcg_at_20 17.628 type value ndcg_at_3 14.536999999999999 type value ndcg_at_5 12.590000000000002 type value precision_at_1 18.0 type value precision_at_10 7.82 type value precision_at_100 1.773 type value precision_at_1000 0.298 type value precision_at_20 5.335 type value precision_at_3 13.5 type value precision_at_5 10.92 type value recall_at_1 3.6580000000000004 type value recall_at_10 15.867999999999999 type value recall_at_100 36.068 type value recall_at_1000 60.608 type value recall_at_20 21.653 type value recall_at_3 8.248 type value recall_at_5 11.108 type Retrieval dataset metrics task config name revision split type default MTEB SICK-E-PL 71bba34b0ece6c56dfcf46d9758a27f7a90f17e9 test PL-MTEB/sicke-pl-pairclassification type value cosine_accuracy 78.12882185079495 type value cosine_accuracy_threshold 95.76345682144165 type value cosine_ap 63.56538407363026 type value cosine_f1 60.88388690639582 type value cosine_f1_threshold 92.86266565322876 type value cosine_precision 49.53104064314426 type value cosine_recall 78.98860398860398 type value dot_accuracy 78.12882185079495 type value dot_accuracy_threshold 95.76345682144165 type value dot_ap 63.56553287602377 type value dot_f1 60.88388690639582 type value dot_f1_threshold 92.86266565322876 type value dot_precision 49.53104064314426 type value dot_recall 78.98860398860398 type value euclidean_accuracy 78.12882185079495 type value euclidean_accuracy_threshold 29.108554124832153 type value euclidean_ap 63.56543484315041 type value euclidean_f1 60.88388690639582 type value euclidean_f1_threshold 37.781822681427 type value euclidean_precision 49.53104064314426 type value euclidean_recall 78.98860398860398 type value main_score 63.56553287602377 type value manhattan_accuracy 77.82307378719935 type value manhattan_accuracy_threshold 658.8656902313232 type value manhattan_ap 63.12761769067177 type value manhattan_f1 60.76436623590872 type value manhattan_f1_threshold 888.3136749267578 type value manhattan_precision 49.48499776085983 type value manhattan_recall 78.70370370370371 type value max_ap 63.56553287602377 type value max_f1 60.88388690639582 type value max_precision 49.53104064314426 type value max_recall 78.98860398860398 type value similarity_accuracy 78.12882185079495 type value similarity_accuracy_threshold 95.76345682144165 type value similarity_ap 63.56538407363026 type value similarity_f1 60.88388690639582 type value similarity_f1_threshold 92.86266565322876 type value similarity_precision 49.53104064314426 type value similarity_recall 78.98860398860398 type PairClassification dataset metrics task config name revision split type default MTEB SICK-R-PL fd5c2441b7eeff8676768036142af4cfa42c1339 test PL-MTEB/sickr-pl-sts type value cosine_pearson 71.75502609028113 type value cosine_spearman 66.52097638938338 type value euclidean_pearson 68.6974439167054 type value euclidean_spearman 66.52095939114172 type value main_score 66.52097638938338 type value manhattan_pearson 68.53848708135571 type value manhattan_spearman 66.29909223435631 type value pearson 71.75502609028113 type value spearman 66.52097638938338 type STS dataset metrics task config name revision split type pl MTEB STS22 (pl) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 40.06621394078099 type value cosine_spearman 45.160446103264285 type value euclidean_pearson 25.38908314629843 type value euclidean_spearman 45.160446103264285 type value main_score 45.160446103264285 type value manhattan_pearson 25.13217941116968 type value manhattan_spearman 45.05397285684081 type value pearson 40.06621394078099 type value spearman 45.160446103264285 type STS dataset metrics task config name revision split type pl-en MTEB STS22 (pl-en) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 77.99167158750629 type value cosine_spearman 77.00326330683939 type value euclidean_pearson 77.60571751826936 type value euclidean_spearman 77.00326330683939 type value main_score 77.00326330683939 type value manhattan_pearson 78.19839585217989 type value manhattan_spearman 78.44894390841364 type value pearson 77.99167158750629 type value spearman 77.00326330683939 type STS dataset metrics task config name revision split type de-pl MTEB STS22 (de-pl) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 40.2221719679774 type value cosine_spearman 57.18465019880842 type value euclidean_pearson 42.11211158455479 type value euclidean_spearman 57.18465019880842 type value main_score 57.18465019880842 type value manhattan_pearson 43.24148614152723 type value manhattan_spearman 56.35320940431847 type value pearson 40.2221719679774 type value spearman 57.18465019880842 type STS dataset metrics task config name revision split type fr-pl MTEB STS22 (fr-pl) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 88.17138238483673 type value cosine_spearman 84.51542547285167 type value euclidean_pearson 87.99782696047525 type value euclidean_spearman 84.51542547285167 type value main_score 84.51542547285167 type value manhattan_pearson 85.811937669563 type value manhattan_spearman 84.51542547285167 type value pearson 88.17138238483673 type value spearman 84.51542547285167 type STS dataset metrics task config name revision split type default MTEB SciFact-PL 47932a35f045ef8ed01ba82bf9ff67f6e109207e test clarin-knext/scifact-pl type value main_score 62.064 type value map_at_1 48.317 type value map_at_10 57.693000000000005 type value map_at_100 58.392999999999994 type value map_at_1000 58.428999999999995 type value map_at_20 58.108000000000004 type value map_at_3 55.293000000000006 type value map_at_5 56.595 type value mrr_at_1 51.0 type value mrr_at_10 59.019576719576705 type value mrr_at_100 59.58007358566797 type value mrr_at_1000 59.61403985820887 type value mrr_at_20 59.35199007075942 type value mrr_at_3 57.166666666666664 type value mrr_at_5 58.08333333333332 type value nauc_map_at_1000_diff1 55.90310480193163 type value nauc_map_at_1000_max 40.922646499130586 type value nauc_map_at_1000_std 6.308307542867231 type value nauc_map_at_100_diff1 55.87923016501095 type value nauc_map_at_100_max 40.930429212300396 type value nauc_map_at_100_std 6.302652510324859 type value nauc_map_at_10_diff1 55.96811326806582 type value nauc_map_at_10_max 40.91912121040118 type value nauc_map_at_10_std 6.315081020792943 type value nauc_map_at_1_diff1 61.615316460538374 type value nauc_map_at_1_max 34.4312789344494 type value nauc_map_at_1_std -2.151749018851701 type value nauc_map_at_20_diff1 55.781940594193316 type value nauc_map_at_20_max 40.877518039008585 type value nauc_map_at_20_std 6.170527123248918 type value nauc_map_at_3_diff1 58.104315292507216 type value nauc_map_at_3_max 39.524635028616544 type value nauc_map_at_3_std 4.367263811245541 type value nauc_map_at_5_diff1 56.60725686218003 type value nauc_map_at_5_max 40.362341129747456 type value nauc_map_at_5_std 5.222556427559299 type value nauc_mrr_at_1000_diff1 56.243518111487454 type value nauc_mrr_at_1000_max 41.92306224416779 type value nauc_mrr_at_1000_std 7.331011181148979 type value nauc_mrr_at_100_diff1 56.21745814714038 type value nauc_mrr_at_100_max 41.92847851363498 type value nauc_mrr_at_100_std 7.322136402819359 type value nauc_mrr_at_10_diff1 56.22224221410973 type value nauc_mrr_at_10_max 42.020110225540144 type value nauc_mrr_at_10_std 7.367785001729785 type value nauc_mrr_at_1_diff1 61.65968884760533 type value nauc_mrr_at_1_max 39.22611274899148 type value nauc_mrr_at_1_std 3.3484556807524357 type value nauc_mrr_at_20_diff1 56.140226618395495 type value nauc_mrr_at_20_max 41.92506913405156 type value nauc_mrr_at_20_std 7.20339996949852 type value nauc_mrr_at_3_diff1 57.82506573973446 type value nauc_mrr_at_3_max 41.962001263558484 type value nauc_mrr_at_3_std 6.909954113302328 type value nauc_mrr_at_5_diff1 56.659054585223565 type value nauc_mrr_at_5_max 42.220145330498326 type value nauc_mrr_at_5_std 6.914754115832333 type value nauc_ndcg_at_1000_diff1 54.101423320176956 type value nauc_ndcg_at_1000_max 42.35761455565217 type value nauc_ndcg_at_1000_std 9.158968107515042 type value nauc_ndcg_at_100_diff1 53.193377266960695 type value nauc_ndcg_at_100_max 42.39818084789296 type value nauc_ndcg_at_100_std 8.982680006715663 type value nauc_ndcg_at_10_diff1 52.7521864873992 type value nauc_ndcg_at_10_max 42.25954681169497 type value nauc_ndcg_at_10_std 9.025856795668409 type value nauc_ndcg_at_1_diff1 61.65968884760533 type value nauc_ndcg_at_1_max 39.22611274899148 type value nauc_ndcg_at_1_std 3.3484556807524357 type value nauc_ndcg_at_20_diff1 52.24054304553779 type value nauc_ndcg_at_20_max 42.14484844258701 type value nauc_ndcg_at_20_std 8.522811774790046 type value nauc_ndcg_at_3_diff1 56.65801023652111 type value nauc_ndcg_at_3_max 41.59901000744857 type value nauc_ndcg_at_3_std 6.866411754213651 type value nauc_ndcg_at_5_diff1 54.25032835371862 type value nauc_ndcg_at_5_max 41.52568005051319 type value nauc_ndcg_at_5_std 6.747184564934237 type value nauc_precision_at_1000_diff1 -12.438995870489618 type value nauc_precision_at_1000_max 33.65458584888833 type value nauc_precision_at_1000_std 38.65000092313945 type value nauc_precision_at_100_diff1 -3.7051397832573696 type value nauc_precision_at_100_max 36.777033924925384 type value nauc_precision_at_100_std 32.24732998272339 type value nauc_precision_at_10_diff1 14.458974499542448 type value nauc_precision_at_10_max 45.75828754327736 type value nauc_precision_at_10_std 31.734511856215665 type value nauc_precision_at_1_diff1 61.65968884760533 type value nauc_precision_at_1_max 39.22611274899148 type value nauc_precision_at_1_std 3.3484556807524357 type value nauc_precision_at_20_diff1 6.911000226020142 type value nauc_precision_at_20_max 42.75953196446269 type value nauc_precision_at_20_std 30.293217657388254 type value nauc_precision_at_3_diff1 39.95888414475174 type value nauc_precision_at_3_max 46.81095681980396 type value nauc_precision_at_3_std 20.732734118894037 type value nauc_precision_at_5_diff1 27.25227607416867 type value nauc_precision_at_5_max 45.278620768210615 type value nauc_precision_at_5_std 22.7094842525771 type value nauc_recall_at_1000_diff1 54.66853408029846 type value nauc_recall_at_1000_max 69.49112978524705 type value nauc_recall_at_1000_std 84.76890756302552 type value nauc_recall_at_100_diff1 33.641140071848085 type value nauc_recall_at_100_max 49.94619316653212 type value nauc_recall_at_100_std 26.970675275760104 type value nauc_recall_at_10_diff1 38.56340942303001 type value nauc_recall_at_10_max 44.13889679913801 type value nauc_recall_at_10_std 17.814455740104584 type value nauc_recall_at_1_diff1 61.615316460538374 type value nauc_recall_at_1_max 34.4312789344494 type value nauc_recall_at_1_std -2.151749018851701 type value nauc_recall_at_20_diff1 33.86997626483988 type value nauc_recall_at_20_max 44.31136705663488 type value nauc_recall_at_20_std 16.58271492635832 type value nauc_recall_at_3_diff1 52.39739118413791 type value nauc_recall_at_3_max 40.56472420414715 type value nauc_recall_at_3_std 7.856902134348368 type value nauc_recall_at_5_diff1 45.693766776717595 type value nauc_recall_at_5_max 41.817545551209086 type value nauc_recall_at_5_std 9.066813773598692 type value ndcg_at_1 51.0 type value ndcg_at_10 62.064 type value ndcg_at_100 65.45 type value ndcg_at_1000 66.366 type value ndcg_at_20 63.418 type value ndcg_at_3 57.915000000000006 type value ndcg_at_5 59.65200000000001 type value precision_at_1 51.0 type value precision_at_10 8.433 type value precision_at_100 1.03 type value precision_at_1000 0.11 type value precision_at_20 4.517 type value precision_at_3 23.0 type value precision_at_5 15.067 type value recall_at_1 48.317 type value recall_at_10 74.078 type value recall_at_100 90.167 type value recall_at_1000 97.333 type value recall_at_20 79.256 type value recall_at_3 62.561 type value recall_at_5 67.039 type Retrieval dataset metrics task config name revision split type default MTEB TRECCOVID-PL 81bcb408f33366c2a20ac54adafad1ae7e877fdd test clarin-knext/trec-covid-pl type value main_score 69.244 type value map_at_1 0.216 type value map_at_10 1.717 type value map_at_100 9.051 type value map_at_1000 21.688 type value map_at_20 2.972 type value map_at_3 0.624 type value map_at_5 0.9809999999999999 type value mrr_at_1 82.0 type value mrr_at_10 88.41666666666666 type value mrr_at_100 88.57051282051282 type value mrr_at_1000 88.57051282051282 type value mrr_at_20 88.57051282051282 type value mrr_at_3 87.66666666666666 type value mrr_at_5 88.16666666666666 type value nauc_map_at_1000_diff1 -21.210172839828886 type value nauc_map_at_1000_max 50.364439193708456 type value nauc_map_at_1000_std 82.23413161215711 type value nauc_map_at_100_diff1 -3.737989437317314 type value nauc_map_at_100_max 40.24314095187729 type value nauc_map_at_100_std 74.6556355692718 type value nauc_map_at_10_diff1 24.069758586207186 type value nauc_map_at_10_max 25.978576944212445 type value nauc_map_at_10_std 30.92185789388276 type value nauc_map_at_1_diff1 33.44422662406722 type value nauc_map_at_1_max 18.58849173002632 type value nauc_map_at_1_std 23.001195148863555 type value nauc_map_at_20_diff1 16.195748164952704 type value nauc_map_at_20_max 32.418991157208055 type value nauc_map_at_20_std 45.053299350375795 type value nauc_map_at_3_diff1 32.94899528110181 type value nauc_map_at_3_max 16.721379232494304 type value nauc_map_at_3_std 18.336699336799814 type value nauc_map_at_5_diff1 30.34930846309755 type value nauc_map_at_5_max 19.37661209832802 type value nauc_map_at_5_std 20.312897662543314 type value nauc_mrr_at_1000_diff1 49.418158929182006 type value nauc_mrr_at_1000_max 67.05328023364747 type value nauc_mrr_at_1000_std 70.85520896614209 type value nauc_mrr_at_100_diff1 49.418158929182006 type value nauc_mrr_at_100_max 67.05328023364747 type value nauc_mrr_at_100_std 70.85520896614209 type value nauc_mrr_at_10_diff1 49.50157932873256 type value nauc_mrr_at_10_max 65.88227845429796 type value nauc_mrr_at_10_std 70.87422352601853 type value nauc_mrr_at_1_diff1 44.82872563057607 type value nauc_mrr_at_1_max 70.45930168520755 type value nauc_mrr_at_1_std 69.88104416785988 type value nauc_mrr_at_20_diff1 49.418158929182006 type value nauc_mrr_at_20_max 67.05328023364747 type value nauc_mrr_at_20_std 70.85520896614209 type value nauc_mrr_at_3_diff1 49.71407489393107 type value nauc_mrr_at_3_max 67.77215590165227 type value nauc_mrr_at_3_std 72.72379898279185 type value nauc_mrr_at_5_diff1 50.328834220772976 type value nauc_mrr_at_5_max 66.34746357369875 type value nauc_mrr_at_5_std 71.51800332961842 type value nauc_ndcg_at_1000_diff1 -11.723371568664843 type value nauc_ndcg_at_1000_max 53.41150083076567 type value nauc_ndcg_at_1000_std 81.94372023908832 type value nauc_ndcg_at_100_diff1 -15.990454633114279 type value nauc_ndcg_at_100_max 45.35431514782352 type value nauc_ndcg_at_100_std 75.73014493320755 type value nauc_ndcg_at_10_diff1 4.30050518239422 type value nauc_ndcg_at_10_max 50.83631607203189 type value nauc_ndcg_at_10_std 63.1087699434136 type value nauc_ndcg_at_1_diff1 17.206529677661354 type value nauc_ndcg_at_1_max 62.14050255620695 type value nauc_ndcg_at_1_std 64.51116243264046 type value nauc_ndcg_at_20_diff1 -5.9182205607515685 type value nauc_ndcg_at_20_max 49.12802457140552 type value nauc_ndcg_at_20_std 68.77672262568693 type value nauc_ndcg_at_3_diff1 22.158007969692125 type value nauc_ndcg_at_3_max 48.17593837968984 type value nauc_ndcg_at_3_std 58.4991887813489 type value nauc_ndcg_at_5_diff1 16.89487399786786 type value nauc_ndcg_at_5_max 46.752900245009414 type value nauc_ndcg_at_5_std 60.870638593862914 type value nauc_precision_at_1000_diff1 -24.67751088399524 type value nauc_precision_at_1000_max 42.70887481946044 type value nauc_precision_at_1000_std 49.219386318590566 type value nauc_precision_at_100_diff1 -19.829901963316278 type value nauc_precision_at_100_max 44.4613898680245 type value nauc_precision_at_100_std 74.8829067578589 type value nauc_precision_at_10_diff1 -0.6759004971171398 type value nauc_precision_at_10_max 52.16154071543153 type value nauc_precision_at_10_std 62.98886080224083 type value nauc_precision_at_1_diff1 44.82872563057607 type value nauc_precision_at_1_max 70.45930168520755 type value nauc_precision_at_1_std 69.88104416785988 type value nauc_precision_at_20_diff1 -11.458671607862547 type value nauc_precision_at_20_max 49.71202888307331 type value nauc_precision_at_20_std 71.79100842422972 type value nauc_precision_at_3_diff1 30.23048096153466 type value nauc_precision_at_3_max 48.24954855245538 type value nauc_precision_at_3_std 54.344575833478935 type value nauc_precision_at_5_diff1 13.925893655561437 type value nauc_precision_at_5_max 46.23506752573775 type value nauc_precision_at_5_std 59.610666544378944 type value nauc_recall_at_1000_diff1 -13.691809447793393 type value nauc_recall_at_1000_max 50.39633577248049 type value nauc_recall_at_1000_std 76.65225154588104 type value nauc_recall_at_100_diff1 4.67778695632382 type value nauc_recall_at_100_max 30.19071079451134 type value nauc_recall_at_100_std 65.03682595699173 type value nauc_recall_at_10_diff1 26.24600831247693 type value nauc_recall_at_10_max 22.235399614875632 type value nauc_recall_at_10_std 27.653841671594176 type value nauc_recall_at_1_diff1 33.44422662406722 type value nauc_recall_at_1_max 18.58849173002632 type value nauc_recall_at_1_std 23.001195148863555 type value nauc_recall_at_20_diff1 19.13211263378722 type value nauc_recall_at_20_max 26.697525172621827 type value nauc_recall_at_20_std 40.9095035359023 type value nauc_recall_at_3_diff1 30.47343886364865 type value nauc_recall_at_3_max 12.854379330237647 type value nauc_recall_at_3_std 14.711252261798258 type value nauc_recall_at_5_diff1 28.344400535065112 type value nauc_recall_at_5_max 14.755638630484144 type value nauc_recall_at_5_std 15.864031786019787 type value ndcg_at_1 72.0 type value ndcg_at_10 69.244 type value ndcg_at_100 50.834 type value ndcg_at_1000 45.535 type value ndcg_at_20 65.676 type value ndcg_at_3 73.776 type value ndcg_at_5 72.715 type value precision_at_1 82.0 type value precision_at_10 73.6 type value precision_at_100 52.22 type value precision_at_1000 20.380000000000003 type value precision_at_20 69.0 type value precision_at_3 81.333 type value precision_at_5 79.2 type value recall_at_1 0.216 type value recall_at_10 1.8900000000000001 type value recall_at_100 12.359 type value recall_at_1000 42.791000000000004 type value recall_at_20 3.44 type value recall_at_3 0.653 type value recall_at_5 1.048 type Retrieval dataset metrics task config name revision split type default MTEB CEDRClassification c0ba03d058e3e1b2f3fd20518875a4563dd12db4 test ai-forever/cedr-classification type value accuracy 43.29968119022317 type value f1 41.112000768648386 type value lrap 72.06216790648348 type value main_score 43.29968119022317 type MultilabelClassification dataset metrics task config name revision split type default MTEB GeoreviewClassification 3765c0d1de6b7d264bc459433c45e5a75513839c test ai-forever/georeview-classification type value accuracy 52.0361328125 type value f1 47.84397823612054 type value f1_weighted 47.84111706041435 type value main_score 52.0361328125 type Classification dataset metrics task config name revision split type default MTEB GeoreviewClusteringP2P 97a313c8fc85b47f13f33e7e9a95c1ad888c7fec test ai-forever/georeview-clustering-p2p type value main_score 60.28266888390485 type value v_measure 60.28266888390485 type value v_measure_std 1.0348363132473835 type Clustering dataset metrics task config name revision split type default MTEB HeadlineClassification 2fe05ee6b5832cda29f2ef7aaad7b7fe6a3609eb test ai-forever/headline-classification type value accuracy 83.4033203125 type value f1 83.39708551274371 type value f1_weighted 83.39502222187862 type value main_score 83.4033203125 type Classification dataset metrics task config name revision split type default MTEB InappropriatenessClassification 601651fdc45ef243751676e62dd7a19f491c0285 test ai-forever/inappropriateness-classification type value accuracy 64.140625 type value ap 59.28880813167948 type value ap_weighted 59.28880813167948 type value f1 63.72032598814496 type value f1_weighted 63.72032598814496 type value main_score 64.140625 type Classification dataset metrics task config name revision split type default MTEB KinopoiskClassification 5911f26666ac11af46cb9c6849d0dc80a378af24 test ai-forever/kinopoisk-sentiment-classification type value accuracy 63.15333333333333 type value f1 59.395986541732384 type value f1_weighted 59.395986541732384 type value main_score 63.15333333333333 type Classification dataset metrics task config name revision split type ru MTEB MIRACLReranking (ru) 6d1962c527217f8927fca80f890f14f36b2802af dev miracl/mmteb-miracl-reranking type value MAP@1(MIRACL) 29.732999999999997 type value MAP@10(MIRACL) 48.333 type value MAP@100(MIRACL) 50.517 type value MAP@1000(MIRACL) 50.517 type value MAP@20(MIRACL) 49.85 type value MAP@3(MIRACL) 41.843 type value MAP@5(MIRACL) 45.323 type value NDCG@1(MIRACL) 48.436 type value NDCG@10(MIRACL) 56.111999999999995 type value NDCG@100(MIRACL) 60.617 type value NDCG@1000(MIRACL) 60.617 type value NDCG@20(MIRACL) 58.826 type value NDCG@3(MIRACL) 50.483999999999995 type value NDCG@5(MIRACL) 52.61 type value P@1(MIRACL) 48.436 type value P@10(MIRACL) 14.667 type value P@100(MIRACL) 1.9529999999999998 type value P@1000(MIRACL) 0.19499999999999998 type value P@20(MIRACL) 8.665000000000001 type value P@3(MIRACL) 31.302000000000003 type value P@5(MIRACL) 23.384 type value Recall@1(MIRACL) 29.732999999999997 type value Recall@10(MIRACL) 66.532 type value Recall@100(MIRACL) 79.952 type value Recall@1000(MIRACL) 79.952 type value Recall@20(MIRACL) 73.75 type value Recall@3(MIRACL) 49.541000000000004 type value Recall@5(MIRACL) 57.389 type value main_score 56.111999999999995 type value nAUC_MAP@1000_diff1(MIRACL) 15.8510181843185 type value nAUC_MAP@1000_max(MIRACL) 27.452155305037095 type value nAUC_MAP@1000_std(MIRACL) 15.147015882448075 type value nAUC_MAP@100_diff1(MIRACL) 15.8510181843185 type value nAUC_MAP@100_max(MIRACL) 27.452155305037095 type value nAUC_MAP@100_std(MIRACL) 15.147015882448075 type value nAUC_MAP@10_diff1(MIRACL) 17.808742699385363 type value nAUC_MAP@10_max(MIRACL) 25.21217663908093 type value nAUC_MAP@10_std(MIRACL) 13.970995033749716 type value nAUC_MAP@1_diff1(MIRACL) 34.30066727981356 type value nAUC_MAP@1_max(MIRACL) 11.096793012814972 type value nAUC_MAP@1_std(MIRACL) 4.298644702770651 type value nAUC_MAP@20_diff1(MIRACL) 16.499957004860978 type value nAUC_MAP@20_max(MIRACL) 26.676987318433714 type value nAUC_MAP@20_std(MIRACL) 15.166175199040485 type value nAUC_MAP@3_diff1(MIRACL) 23.797870452650084 type value nAUC_MAP@3_max(MIRACL) 18.20460307122738 type value nAUC_MAP@3_std(MIRACL) 8.985118628338126 type value nAUC_MAP@5_diff1(MIRACL) 20.549029352694866 type value nAUC_MAP@5_max(MIRACL) 21.528805328834324 type value nAUC_MAP@5_std(MIRACL) 11.131951589460492 type value nAUC_NDCG@1000_diff1(MIRACL) 5.973372149854828 type value nAUC_NDCG@1000_max(MIRACL) 36.70565868748619 type value nAUC_NDCG@1000_std(MIRACL) 19.551007976769245 type value nAUC_NDCG@100_diff1(MIRACL) 5.973372149854828 type value nAUC_NDCG@100_max(MIRACL) 36.70565868748619 type value nAUC_NDCG@100_std(MIRACL) 19.551007976769245 type value nAUC_NDCG@10_diff1(MIRACL) 10.894100451667919 type value nAUC_NDCG@10_max(MIRACL) 31.735109695399416 type value nAUC_NDCG@10_std(MIRACL) 17.674556265190706 type value nAUC_NDCG@1_diff1(MIRACL) 22.04892839322977 type value nAUC_NDCG@1_max(MIRACL) 32.51034181981298 type value nAUC_NDCG@1_std(MIRACL) 14.343760356007765 type value nAUC_NDCG@20_diff1(MIRACL) 8.074119776676103 type value nAUC_NDCG@20_max(MIRACL) 34.52221220694718 type value nAUC_NDCG@20_std(MIRACL) 19.94006423667 type value nAUC_NDCG@3_diff1(MIRACL) 16.284195830367825 type value nAUC_NDCG@3_max(MIRACL) 26.521965826220352 type value nAUC_NDCG@3_std(MIRACL) 13.850033289666094 type value nAUC_NDCG@5_diff1(MIRACL) 14.362693198633952 type value nAUC_NDCG@5_max(MIRACL) 27.781809390068872 type value nAUC_NDCG@5_std(MIRACL) 14.879808284537981 type value nAUC_P@1000_diff1(MIRACL) -27.606682296231373 type value nAUC_P@1000_max(MIRACL) 33.03084251491326 type value nAUC_P@1000_std(MIRACL) 15.674013757663898 type value nAUC_P@100_diff1(MIRACL) -27.606682296231327 type value nAUC_P@100_max(MIRACL) 33.03084251491332 type value nAUC_P@100_std(MIRACL) 15.674013757663937 type value nAUC_P@10_diff1(MIRACL) -23.575685602922174 type value nAUC_P@10_max(MIRACL) 36.72548498655645 type value nAUC_P@10_std(MIRACL) 21.317694028285104 type value nAUC_P@1_diff1(MIRACL) 22.04892839322977 type value nAUC_P@1_max(MIRACL) 32.51034181981298 type value nAUC_P@1_std(MIRACL) 14.343760356007765 type value nAUC_P@20_diff1(MIRACL) -26.064734965649322 type value nAUC_P@20_max(MIRACL) 34.10936682680113 type value nAUC_P@20_std(MIRACL) 20.31615496254574 type value nAUC_P@3_diff1(MIRACL) -10.903444655544746 type value nAUC_P@3_max(MIRACL) 34.33585029049373 type value nAUC_P@3_std(MIRACL) 18.620142249622834 type value nAUC_P@5_diff1(MIRACL) -18.454884144221385 type value nAUC_P@5_max(MIRACL) 35.620428961110036 type value nAUC_P@5_std(MIRACL) 20.265460635926893 type value nAUC_Recall@1000_diff1(MIRACL) -28.25716669219796 type value nAUC_Recall@1000_max(MIRACL) 59.88673755432144 type value nAUC_Recall@1000_std(MIRACL) 29.916576785101622 type value nAUC_Recall@100_diff1(MIRACL) -28.25716669219796 type value nAUC_Recall@100_max(MIRACL) 59.88673755432144 type value nAUC_Recall@100_std(MIRACL) 29.916576785101622 type value nAUC_Recall@10_diff1(MIRACL) -2.5731369116803466 type value nAUC_Recall@10_max(MIRACL) 34.37108435281944 type value nAUC_Recall@10_std(MIRACL) 20.744457001608925 type value nAUC_Recall@1_diff1(MIRACL) 34.30066727981356 type value nAUC_Recall@1_max(MIRACL) 11.096793012814972 type value nAUC_Recall@1_std(MIRACL) 4.298644702770651 type value nAUC_Recall@20_diff1(MIRACL) -13.667980220614172 type value nAUC_Recall@20_max(MIRACL) 44.947659106700044 type value nAUC_Recall@20_std(MIRACL) 29.413435369376923 type value nAUC_Recall@3_diff1(MIRACL) 15.838199908854786 type value nAUC_Recall@3_max(MIRACL) 17.368565662731196 type value nAUC_Recall@3_std(MIRACL) 10.538072940876807 type value nAUC_Recall@5_diff1(MIRACL) 8.199967584892176 type value nAUC_Recall@5_max(MIRACL) 23.500985460573578 type value nAUC_Recall@5_std(MIRACL) 13.477424183539433 type Reranking dataset metrics task config name revision split type ru MTEB MIRACLRetrieval (ru) main dev miracl/mmteb-miracl type value main_score 52.211 type value map_at_1 23.238 type value map_at_10 41.559000000000005 type value map_at_100 44.757999999999996 type value map_at_1000 44.861000000000004 type value map_at_20 43.461 type value map_at_3 34.593 type value map_at_5 38.056 type value mrr_at_1 47.04472843450479 type value mrr_at_10 59.587485420153286 type value mrr_at_100 60.17662556783717 type value mrr_at_1000 60.1850174860852 type value mrr_at_20 60.003979383733544 type value mrr_at_3 56.62939297124608 type value mrr_at_5 58.33067092651768 type value nauc_map_at_1000_diff1 26.665139374258256 type value nauc_map_at_1000_max 20.20801190375824 type value nauc_map_at_1000_std 3.35434510540552 type value nauc_map_at_100_diff1 26.691816652639787 type value nauc_map_at_100_max 20.193510183457917 type value nauc_map_at_100_std 3.371679544337864 type value nauc_map_at_10_diff1 27.24904607990151 type value nauc_map_at_10_max 18.26589731339405 type value nauc_map_at_10_std 1.0177924180874538 type value nauc_map_at_1_diff1 34.53595808193455 type value nauc_map_at_1_max 10.970155439499656 type value nauc_map_at_1_std -3.8626873246816373 type value nauc_map_at_20_diff1 26.8513788979128 type value nauc_map_at_20_max 19.367475736662428 type value nauc_map_at_20_std 2.2475091146613564 type value nauc_map_at_3_diff1 28.911815196615866 type value nauc_map_at_3_max 15.474121149651292 type value nauc_map_at_3_std -1.0664535264565158 type value nauc_map_at_5_diff1 27.772031743222787 type value nauc_map_at_5_max 16.241638808384145 type value nauc_map_at_5_std -0.6044307972013538 type value nauc_mrr_at_1000_diff1 26.66563442138901 type value nauc_mrr_at_1000_max 27.74734004586503 type value nauc_mrr_at_1000_std 10.663042801330587 type value nauc_mrr_at_100_diff1 26.66809693875436 type value nauc_mrr_at_100_max 27.7565667281779 type value nauc_mrr_at_100_std 10.671838040923266 type value nauc_mrr_at_10_diff1 26.587658592417736 type value nauc_mrr_at_10_max 27.872712998242328 type value nauc_mrr_at_10_std 10.979716151856918 type value nauc_mrr_at_1_diff1 29.30751401472168 type value nauc_mrr_at_1_max 24.98212676568516 type value nauc_mrr_at_1_std 6.094206809391165 type value nauc_mrr_at_20_diff1 26.52396413399926 type value nauc_mrr_at_20_max 27.720568784204847 type value nauc_mrr_at_20_std 10.749903126459412 type value nauc_mrr_at_3_diff1 26.993782403961802 type value nauc_mrr_at_3_max 27.810128603605342 type value nauc_mrr_at_3_std 10.526250026174825 type value nauc_mrr_at_5_diff1 26.491056284663404 type value nauc_mrr_at_5_max 27.938292238745838 type value nauc_mrr_at_5_std 10.620036152236098 type value nauc_ndcg_at_1000_diff1 24.743263734342236 type value nauc_ndcg_at_1000_max 25.632023742967196 type value nauc_ndcg_at_1000_std 9.54979482991325 type value nauc_ndcg_at_100_diff1 24.884477288371073 type value nauc_ndcg_at_100_max 25.856099754401797 type value nauc_ndcg_at_100_std 10.275002448873611 type value nauc_ndcg_at_10_diff1 25.813663674330005 type value nauc_ndcg_at_10_max 21.4632558325771 type value nauc_ndcg_at_10_std 4.793772488457711 type value nauc_ndcg_at_1_diff1 29.30751401472168 type value nauc_ndcg_at_1_max 24.98212676568516 type value nauc_ndcg_at_1_std 6.094206809391165 type value nauc_ndcg_at_20_diff1 24.96712085611002 type value nauc_ndcg_at_20_max 23.176681160212546 type value nauc_ndcg_at_20_std 6.936886476037671 type value nauc_ndcg_at_3_diff1 25.475637018641205 type value nauc_ndcg_at_3_max 22.040672063815855 type value nauc_ndcg_at_3_std 5.327531594448605 type value nauc_ndcg_at_5_diff1 25.70702625003538 type value nauc_ndcg_at_5_max 20.273499330943313 type value nauc_ndcg_at_5_std 3.733783938564952 type value nauc_precision_at_1000_diff1 -14.918023025551047 type value nauc_precision_at_1000_max 18.668936317187704 type value nauc_precision_at_1000_std 19.15643973163778 type value nauc_precision_at_100_diff1 -12.902497092152561 type value nauc_precision_at_100_max 22.117700522212857 type value nauc_precision_at_100_std 23.367379142816734 type value nauc_precision_at_10_diff1 -3.319884895143968 type value nauc_precision_at_10_max 25.207453700919412 type value nauc_precision_at_10_std 16.768944029523773 type value nauc_precision_at_1_diff1 29.30751401472168 type value nauc_precision_at_1_max 24.98212676568516 type value nauc_precision_at_1_std 6.094206809391165 type value nauc_precision_at_20_diff1 -8.101925051455304 type value nauc_precision_at_20_max 23.93155685736234 type value nauc_precision_at_20_std 19.599852197885983 type value nauc_precision_at_3_diff1 8.604157546918138 type value nauc_precision_at_3_max 26.8274074367336 type value nauc_precision_at_3_std 13.210078569814973 type value nauc_precision_at_5_diff1 2.0240126571446004 type value nauc_precision_at_5_max 25.068271323836683 type value nauc_precision_at_5_std 13.423044252359 type value nauc_recall_at_1000_diff1 2.5057442905176264 type value nauc_recall_at_1000_max 57.765040045333485 type value nauc_recall_at_1000_std 75.40225417846978 type value nauc_recall_at_100_diff1 13.982399962667946 type value nauc_recall_at_100_max 36.06499090419987 type value nauc_recall_at_100_std 38.55877836909554 type value nauc_recall_at_10_diff1 19.09907433139298 type value nauc_recall_at_10_max 14.320755651797818 type value nauc_recall_at_10_std 3.68835109545608 type value nauc_recall_at_1_diff1 34.53595808193455 type value nauc_recall_at_1_max 10.970155439499656 type value nauc_recall_at_1_std -3.8626873246816373 type value nauc_recall_at_20_diff1 15.80854510984775 type value nauc_recall_at_20_max 17.20627614536354 type value nauc_recall_at_20_std 9.028188051323042 type value nauc_recall_at_3_diff1 23.88853757885772 type value nauc_recall_at_3_max 13.29954353582913 type value nauc_recall_at_3_std -0.42190904806759966 type value nauc_recall_at_5_diff1 20.720312115028822 type value nauc_recall_at_5_max 12.324541527710025 type value nauc_recall_at_5_std -0.19420222400103399 type value ndcg_at_1 47.044999999999995 type value ndcg_at_10 52.211 type value ndcg_at_100 60.777 type value ndcg_at_1000 61.951 type value ndcg_at_20 56.215 type value ndcg_at_3 45.871 type value ndcg_at_5 47.643 type value precision_at_1 47.044999999999995 type value precision_at_10 16.062 type value precision_at_100 2.563 type value precision_at_1000 0.27899999999999997 type value precision_at_20 9.9 type value precision_at_3 31.575999999999997 type value precision_at_5 24.153 type value recall_at_1 23.238 type value recall_at_10 63.479 type value recall_at_100 91.51899999999999 type value recall_at_1000 97.906 type value recall_at_20 74.705 type value recall_at_3 42.082 type value recall_at_5 50.708 type Retrieval dataset metrics task config name revision split type ru MTEB MassiveIntentClassification (ru) 4672e20407010da34463acc759c162ca9734bca6 test mteb/amazon_massive_intent type value accuracy 68.61466039004706 type value f1 63.790707574282045 type value f1_weighted 67.28456899088164 type value main_score 68.61466039004706 type Classification dataset metrics task config name revision split type ru MTEB MassiveScenarioClassification (ru) fad2c6e8459f9e1c45d9315f4953d921437d70f8 test mteb/amazon_massive_scenario type value accuracy 77.97579018157364 type value f1 76.31497051309336 type value f1_weighted 77.54198422119202 type value main_score 77.97579018157364 type Classification dataset metrics task config name revision split type default MTEB RUParaPhraserSTS 43265056790b8f7c59e0139acb4be0a8dad2c8f4 test merionum/ru_paraphraser type value cosine_pearson 62.072853635744465 type value cosine_spearman 68.32627155640247 type value euclidean_pearson 65.56072460948485 type value euclidean_spearman 68.32632364995054 type value main_score 68.32627155640247 type value manhattan_pearson 65.54799770948776 type value manhattan_spearman 68.2428132570697 type value pearson 62.072853635744465 type value spearman 68.32627155640247 type STS dataset metrics task config name revision split type default MTEB RiaNewsRetrieval 82374b0bbacda6114f39ff9c5b925fa1512ca5d7 test ai-forever/ria-news-retrieval type value main_score 79.42399999999999 type value map_at_1 67.42 type value map_at_10 75.81700000000001 type value map_at_100 76.103 type value map_at_1000 76.11099999999999 type value map_at_20 76.011 type value map_at_3 74.38 type value map_at_5 75.31400000000001 type value mrr_at_1 67.42 type value mrr_at_10 75.81702380952322 type value mrr_at_100 76.10294206257022 type value mrr_at_1000 76.11127333184083 type value mrr_at_20 76.01092756817413 type value mrr_at_3 74.37999999999947 type value mrr_at_5 75.31449999999931 type value nauc_map_at_1000_diff1 74.47312749692254 type value nauc_map_at_1000_max 24.255650636762592 type value nauc_map_at_1000_std -13.538045103707466 type value nauc_map_at_100_diff1 74.46935527123232 type value nauc_map_at_100_max 24.260637479032273 type value nauc_map_at_100_std -13.526893488105108 type value nauc_map_at_10_diff1 74.37904649319015 type value nauc_map_at_10_max 24.25477514829031 type value nauc_map_at_10_std -13.673101053529166 type value nauc_map_at_1_diff1 77.41742450291483 type value nauc_map_at_1_max 21.561634939014 type value nauc_map_at_1_std -15.302925641163046 type value nauc_map_at_20_diff1 74.44339113303336 type value nauc_map_at_20_max 24.281346979231508 type value nauc_map_at_20_std -13.533874833150467 type value nauc_map_at_3_diff1 74.31017752460161 type value nauc_map_at_3_max 24.209272036097506 type value nauc_map_at_3_std -14.053104049162751 type value nauc_map_at_5_diff1 74.42859541067173 type value nauc_map_at_5_max 24.16570861589971 type value nauc_map_at_5_std -13.948432311463257 type value nauc_mrr_at_1000_diff1 74.47312785315074 type value nauc_mrr_at_1000_max 24.255652429274488 type value nauc_mrr_at_1000_std -13.538043692357599 type value nauc_mrr_at_100_diff1 74.46935527123232 type value nauc_mrr_at_100_max 24.260637479032273 type value nauc_mrr_at_100_std -13.526893488105108 type value nauc_mrr_at_10_diff1 74.37904649319015 type value nauc_mrr_at_10_max 24.25477514829031 type value nauc_mrr_at_10_std -13.673101053529166 type value nauc_mrr_at_1_diff1 77.41742450291483 type value nauc_mrr_at_1_max 21.561634939014 type value nauc_mrr_at_1_std -15.302925641163046 type value nauc_mrr_at_20_diff1 74.44339113303336 type value nauc_mrr_at_20_max 24.281346979231508 type value nauc_mrr_at_20_std -13.533874833150467 type value nauc_mrr_at_3_diff1 74.31017752460161 type value nauc_mrr_at_3_max 24.209272036097506 type value nauc_mrr_at_3_std -14.053104049162751 type value nauc_mrr_at_5_diff1 74.42859541067173 type value nauc_mrr_at_5_max 24.16570861589971 type value nauc_mrr_at_5_std -13.948432311463257 type value nauc_ndcg_at_1000_diff1 73.67049349073889 type value nauc_ndcg_at_1000_max 25.36219767677513 type value nauc_ndcg_at_1000_std -12.018149673769434 type value nauc_ndcg_at_100_diff1 73.52540106541404 type value nauc_ndcg_at_100_max 25.54104779422804 type value nauc_ndcg_at_100_std -11.596858470683141 type value nauc_ndcg_at_10_diff1 73.13668875552696 type value nauc_ndcg_at_10_max 25.555285618887662 type value nauc_ndcg_at_10_std -12.31485256997023 type value nauc_ndcg_at_1_diff1 77.41742450291483 type value nauc_ndcg_at_1_max 21.561634939014 type value nauc_ndcg_at_1_std -15.302925641163046 type value nauc_ndcg_at_20_diff1 73.35771732216482 type value nauc_ndcg_at_20_max 25.73112191366883 type value nauc_ndcg_at_20_std -11.69854261340669 type value nauc_ndcg_at_3_diff1 73.20274751289709 type value nauc_ndcg_at_3_max 25.285529084214925 type value nauc_ndcg_at_3_std -13.37770120862227 type value nauc_ndcg_at_5_diff1 73.33594229336342 type value nauc_ndcg_at_5_max 25.281830078361644 type value nauc_ndcg_at_5_std -13.088615162069974 type value nauc_precision_at_1000_diff1 55.90120106013352 type value nauc_precision_at_1000_max 55.70083105705886 type value nauc_precision_at_1000_std 36.2217350708384 type value nauc_precision_at_100_diff1 59.2870776629234 type value nauc_precision_at_100_max 47.133189559008834 type value nauc_precision_at_100_std 28.301920571571802 type value nauc_precision_at_10_diff1 65.12757705051081 type value nauc_precision_at_10_max 34.0578425463014 type value nauc_precision_at_10_std -2.7826038995063618 type value nauc_precision_at_1_diff1 77.41742450291483 type value nauc_precision_at_1_max 21.561634939014 type value nauc_precision_at_1_std -15.302925641163046 type value nauc_precision_at_20_diff1 64.13592064044578 type value nauc_precision_at_20_max 39.3269437952694 type value nauc_precision_at_20_std 7.181669511985859 type value nauc_precision_at_3_diff1 68.88283614651107 type value nauc_precision_at_3_max 29.546078723110387 type value nauc_precision_at_3_std -10.635148066667597 type value nauc_precision_at_5_diff1 68.11610612745827 type value nauc_precision_at_5_max 30.708733892411683 type value nauc_precision_at_5_std -8.722606142068399 type value nauc_recall_at_1000_diff1 55.90120106013372 type value nauc_recall_at_1000_max 55.70083105705975 type value nauc_recall_at_1000_std 36.22173507083937 type value nauc_recall_at_100_diff1 59.287077662923856 type value nauc_recall_at_100_max 47.1331895590096 type value nauc_recall_at_100_std 28.30192057157174 type value nauc_recall_at_10_diff1 65.1275770505108 type value nauc_recall_at_10_max 34.057842546301245 type value nauc_recall_at_10_std -2.7826038995065376 type value nauc_recall_at_1_diff1 77.41742450291483 type value nauc_recall_at_1_max 21.561634939014 type value nauc_recall_at_1_std -15.302925641163046 type value nauc_recall_at_20_diff1 64.13592064044556 type value nauc_recall_at_20_max 39.32694379526965 type value nauc_recall_at_20_std 7.181669511986287 type value nauc_recall_at_3_diff1 68.88283614651114 type value nauc_recall_at_3_max 29.54607872311032 type value nauc_recall_at_3_std -10.635148066667742 type value nauc_recall_at_5_diff1 68.11610612745811 type value nauc_recall_at_5_max 30.70873389241151 type value nauc_recall_at_5_std -8.722606142068207 type value ndcg_at_1 67.42 type value ndcg_at_10 79.42399999999999 type value ndcg_at_100 80.754 type value ndcg_at_1000 80.979 type value ndcg_at_20 80.118 type value ndcg_at_3 76.543 type value ndcg_at_5 78.215 type value precision_at_1 67.42 type value precision_at_10 9.052 type value precision_at_100 0.966 type value precision_at_1000 0.098 type value precision_at_20 4.662 type value precision_at_3 27.589999999999996 type value precision_at_5 17.36 type value recall_at_1 67.42 type value recall_at_10 90.52 type value recall_at_100 96.61 type value recall_at_1000 98.39 type value recall_at_20 93.24 type value recall_at_3 82.77 type value recall_at_5 86.8 type Retrieval dataset metrics task config name revision split type default MTEB RuBQReranking 2e96b8f098fa4b0950fc58eacadeb31c0d0c7fa2 test ai-forever/rubq-reranking type value main_score 68.48180892753541 type value map 68.48180892753541 type value mrr 73.69372550223615 type value nAUC_map_diff1 37.93778560797301 type value nAUC_map_max 10.858022431340633 type value nAUC_map_std 6.446466714820493 type value nAUC_mrr_diff1 39.83698029227208 type value nAUC_mrr_max 14.378309445768284 type value nAUC_mrr_std 10.579567761464919 type Reranking dataset metrics task config name revision split type default MTEB RuBQRetrieval e19b6ffa60b3bc248e0b41f4cc37c26a55c2a67b test ai-forever/rubq-retrieval type value main_score 66.77 type value map_at_1 36.525 type value map_at_10 58.021 type value map_at_100 59.016000000000005 type value map_at_1000 59.041999999999994 type value map_at_20 58.709 type value map_at_3 51.808 type value map_at_5 55.706999999999994 type value mrr_at_1 52.95508274231678 type value mrr_at_10 66.10029926076034 type value mrr_at_100 66.46489903689454 type value mrr_at_1000 66.47135430048212 type value mrr_at_20 66.36282360130573 type value mrr_at_3 63.347123719464236 type value mrr_at_5 65.20291568163925 type value nauc_map_at_1000_diff1 36.39353112777031 type value nauc_map_at_1000_max 14.511234479555156 type value nauc_map_at_1000_std -12.003784393055856 type value nauc_map_at_100_diff1 36.396297354858326 type value nauc_map_at_100_max 14.532932252459755 type value nauc_map_at_100_std -11.9933713072409 type value nauc_map_at_10_diff1 36.19731963995984 type value nauc_map_at_10_max 14.331593327284844 type value nauc_map_at_10_std -12.607001882190588 type value nauc_map_at_1_diff1 39.04224394212489 type value nauc_map_at_1_max 9.44079807509392 type value nauc_map_at_1_std -8.725551038382205 type value nauc_map_at_20_diff1 36.27250811060138 type value nauc_map_at_20_max 14.521970331255876 type value nauc_map_at_20_std -12.033391150828098 type value nauc_map_at_3_diff1 35.966460233965485 type value nauc_map_at_3_max 11.62955834976298 type value nauc_map_at_3_std -13.649024048480133 type value nauc_map_at_5_diff1 36.131815002934644 type value nauc_map_at_5_max 13.157509275481777 type value nauc_map_at_5_std -13.36839170298778 type value nauc_mrr_at_1000_diff1 40.191647456610056 type value nauc_mrr_at_1000_max 16.63142892913043 type value nauc_mrr_at_1000_std -12.671951113868769 type value nauc_mrr_at_100_diff1 40.18726742271696 type value nauc_mrr_at_100_max 16.638314382103207 type value nauc_mrr_at_100_std -12.664912420744438 type value nauc_mrr_at_10_diff1 40.028293277796855 type value nauc_mrr_at_10_max 16.841638035795718 type value nauc_mrr_at_10_std -12.781785759758687 type value nauc_mrr_at_1_diff1 42.26303997344821 type value nauc_mrr_at_1_max 14.211014905785252 type value nauc_mrr_at_1_std -11.030701637062437 type value nauc_mrr_at_20_diff1 40.12680433695074 type value nauc_mrr_at_20_max 16.75915749592042 type value nauc_mrr_at_20_std -12.613807048523782 type value nauc_mrr_at_3_diff1 40.32434278687767 type value nauc_mrr_at_3_max 15.811615950737387 type value nauc_mrr_at_3_std -13.957860180387636 type value nauc_mrr_at_5_diff1 40.09422159913817 type value nauc_mrr_at_5_max 16.64090259238879 type value nauc_mrr_at_5_std -13.230746065794726 type value nauc_ndcg_at_1000_diff1 36.67352791454268 type value nauc_ndcg_at_1000_max 16.749915190801016 type value nauc_ndcg_at_1000_std -11.008545008175378 type value nauc_ndcg_at_100_diff1 36.58072887287039 type value nauc_ndcg_at_100_max 17.22374718832945 type value nauc_ndcg_at_100_std -10.559637745205016 type value nauc_ndcg_at_10_diff1 35.786024269753334 type value nauc_ndcg_at_10_max 17.217091860749864 type value nauc_ndcg_at_10_std -12.505927857541066 type value nauc_ndcg_at_1_diff1 42.41055520049291 type value nauc_ndcg_at_1_max 14.001922648893919 type value nauc_ndcg_at_1_std -11.224085018036103 type value nauc_ndcg_at_20_diff1 35.9577978619838 type value nauc_ndcg_at_20_max 17.612142353807204 type value nauc_ndcg_at_20_std -10.715656533623179 type value nauc_ndcg_at_3_diff1 35.92331458170165 type value nauc_ndcg_at_3_max 12.972908846104833 type value nauc_ndcg_at_3_std -14.90499944816046 type value nauc_ndcg_at_5_diff1 35.87509174776851 type value nauc_ndcg_at_5_max 15.016606655112842 type value nauc_ndcg_at_5_std -14.252766370474959 type value nauc_precision_at_1000_diff1 -7.854237065573715 type value nauc_precision_at_1000_max 7.340193640831781 type value nauc_precision_at_1000_std 5.270139452495764 type value nauc_precision_at_100_diff1 -5.433762342336105 type value nauc_precision_at_100_max 10.323131724715576 type value nauc_precision_at_100_std 6.065361232063088 type value nauc_precision_at_10_diff1 1.6163013309854788 type value nauc_precision_at_10_max 13.853149437703955 type value nauc_precision_at_10_std -0.4630873244645538 type value nauc_precision_at_1_diff1 42.41055520049291 type value nauc_precision_at_1_max 14.001922648893919 type value nauc_precision_at_1_std -11.224085018036103 type value nauc_precision_at_20_diff1 -2.406608082278331 type value nauc_precision_at_20_max 12.672408320017443 type value nauc_precision_at_20_std 4.420612595577876 type value nauc_precision_at_3_diff1 15.724555799730243 type value nauc_precision_at_3_max 12.818558415088615 type value nauc_precision_at_3_std -11.49979730611224 type value nauc_precision_at_5_diff1 8.485573750280292 type value nauc_precision_at_5_max 13.304773839372094 type value nauc_precision_at_5_std -6.633911950881821 type value nauc_recall_at_1000_diff1 -7.902591492154048 type value nauc_recall_at_1000_max 54.202835032879946 type value nauc_recall_at_1000_std 68.22401286555711 type value nauc_recall_at_100_diff1 14.88281690495126 type value nauc_recall_at_100_max 41.9305338281276 type value nauc_recall_at_100_std 30.260295038603324 type value nauc_recall_at_10_diff1 23.09613458762812 type value nauc_recall_at_10_max 24.921985669652386 type value nauc_recall_at_10_std -9.990910822464661 type value nauc_recall_at_1_diff1 39.04224394212489 type value nauc_recall_at_1_max 9.44079807509392 type value nauc_recall_at_1_std -8.725551038382205 type value nauc_recall_at_20_diff1 19.41298369752395 type value nauc_recall_at_20_max 31.91169321346991 type value nauc_recall_at_20_std 4.514353181881159 type value nauc_recall_at_3_diff1 29.514018426239197 type value nauc_recall_at_3_max 10.600179069626673 type value nauc_recall_at_3_std -17.02685998662361 type value nauc_recall_at_5_diff1 26.66966838912029 type value nauc_recall_at_5_max 15.359436829533934 type value nauc_recall_at_5_std -15.87666175175801 type value ndcg_at_1 52.896 type value ndcg_at_10 66.77 type value ndcg_at_100 69.98100000000001 type value ndcg_at_1000 70.408 type value ndcg_at_20 68.53200000000001 type value ndcg_at_3 58.074999999999996 type value ndcg_at_5 62.841 type value precision_at_1 52.896 type value precision_at_10 13.8 type value precision_at_100 1.609 type value precision_at_1000 0.166 type value precision_at_20 7.444000000000001 type value precision_at_3 32.623999999999995 type value precision_at_5 23.735 type value recall_at_1 36.525 type value recall_at_10 83.893 type value recall_at_100 96.345 type value recall_at_1000 99.126 type value recall_at_20 89.812 type value recall_at_3 62.58899999999999 type value recall_at_5 73.64500000000001 type Retrieval dataset metrics task config name revision split type default MTEB RuReviewsClassification f6d2c31f4dc6b88f468552750bfec05b4b41b05a test ai-forever/ru-reviews-classification type value accuracy 68.2373046875 type value f1 66.6798984937843 type value f1_weighted 66.67858774240374 type value main_score 68.2373046875 type Classification dataset metrics task config name revision split type default MTEB RuSTSBenchmarkSTS 7cf24f325c6da6195df55bef3d86b5e0616f3018 test ai-forever/ru-stsbenchmark-sts type value cosine_pearson 77.06911833905438 type value cosine_spearman 77.84605139753621 type value euclidean_pearson 76.3616511204864 type value euclidean_spearman 77.84487946345095 type value main_score 77.84605139753621 type value manhattan_pearson 76.35303659263998 type value manhattan_spearman 77.87677782965115 type value pearson 77.06911833905438 type value spearman 77.84605139753621 type STS dataset metrics task config name revision split type default MTEB RuSciBenchGRNTIClassification 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1 test ai-forever/ru-scibench-grnti-classification type value accuracy 61.23535156249999 type value f1 59.029291161802334 type value f1_weighted 59.041548793589406 type value main_score 61.23535156249999 type Classification dataset metrics task config name revision split type default MTEB RuSciBenchGRNTIClusteringP2P 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1 test ai-forever/ru-scibench-grnti-classification type value main_score 56.82815630686135 type value v_measure 56.82815630686135 type value v_measure_std 0.6871068462323323 type Clustering dataset metrics task config name revision split type default MTEB RuSciBenchOECDClassification 26c88e99dcaba32bb45d0e1bfc21902337f6d471 test ai-forever/ru-scibench-oecd-classification type value accuracy 48.1005859375 type value f1 44.918516110124315 type value f1_weighted 44.91942618115105 type value main_score 48.1005859375 type Classification dataset metrics task config name revision split type default MTEB RuSciBenchOECDClusteringP2P 26c88e99dcaba32bb45d0e1bfc21902337f6d471 test ai-forever/ru-scibench-oecd-classification type value main_score 48.72707742931753 type value v_measure 48.72707742931753 type value v_measure_std 0.7258468439420995 type Clustering dataset metrics task config name revision split type ru MTEB STS22 (ru) de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 test mteb/sts22-crosslingual-sts type value cosine_pearson 64.95220904597029 type value cosine_spearman 67.35282990065247 type value euclidean_pearson 64.72045496418937 type value euclidean_spearman 67.35282990065247 type value main_score 67.35282990065247 type value manhattan_pearson 64.40621455763392 type value manhattan_spearman 66.99408273892949 type value pearson 64.95220904597029 type value spearman 67.35282990065247 type STS dataset metrics task config name revision split type default MTEB SensitiveTopicsClassification 416b34a802308eac30e4192afc0ff99bb8dcc7f2 test ai-forever/sensitive-topics-classification type value accuracy 29.624023437500004 type value f1 33.214028020582894 type value lrap 44.53599717881868 type value main_score 29.624023437500004 type MultilabelClassification dataset metrics task config name revision split type default MTEB TERRa 7b58f24536063837d644aab9a023c62199b2a612 dev ai-forever/terra-pairclassification type value cosine_accuracy 57.98045602605863 type value cosine_accuracy_threshold 83.04829597473145 type value cosine_ap 55.56580974377611 type value cosine_f1 66.9603524229075 type value cosine_f1_threshold 73.216313123703 type value cosine_precision 50.498338870431894 type value cosine_recall 99.34640522875817 type value dot_accuracy 57.98045602605863 type value dot_accuracy_threshold 83.04829597473145 type value dot_ap 55.56580974377611 type value dot_f1 66.9603524229075 type value dot_f1_threshold 73.21631908416748 type value dot_precision 50.498338870431894 type value dot_recall 99.34640522875817 type value euclidean_accuracy 57.98045602605863 type value euclidean_accuracy_threshold 58.226633071899414 type value euclidean_ap 55.56580974377611 type value euclidean_f1 66.9603524229075 type value euclidean_f1_threshold 73.18969368934631 type value euclidean_precision 50.498338870431894 type value euclidean_recall 99.34640522875817 type value main_score 55.56580974377611 type value manhattan_accuracy 57.98045602605863 type value manhattan_accuracy_threshold 1336.6012573242188 type value manhattan_ap 55.5371135438789 type value manhattan_f1 66.95842450765863 type value manhattan_f1_threshold 1720.5078125 type value manhattan_precision 50.32894736842105 type value manhattan_recall 100.0 type value max_ap 55.56580974377611 type value max_f1 66.9603524229075 type value max_precision 50.498338870431894 type value max_recall 100.0 type value similarity_accuracy 57.98045602605863 type value similarity_accuracy_threshold 83.04829597473145 type value similarity_ap 55.56580974377611 type value similarity_f1 66.9603524229075 type value similarity_f1_threshold 73.216313123703 type value similarity_precision 50.498338870431894 type value similarity_recall 99.34640522875817 type PairClassification	sentence-similarity	sentence-transformers feature-extraction sentence-similarity mteb

KaLM-Embedding

KaLM-Embedding is a series of embedding models adapted from auto-regressive LLMs with superior training data. KaLM-embedding-multilingual-mini is trained from Qwen/Qwen2-0.5B with massive weakly-supervised pre-training and supervised fine-tuning data. ## 📑 Open-source Plan - [x] Model Checkpoint - [x] KaLM-embedding-multilingual-mini-v1 - [x] KaLM-embedding-multilingual-mini-instruct-v1 - [ ] KaLM-embedding-multilingual-max-v1 - [x] Training and Evaluation Code: HITsz-TMG/KaLM-Embedding - [ ] Technical Report (in 2024) - [ ] Training Data ## Evaluation | Model Name | Model Size | C-MTEB(35) | MTEB(56) | avg |:----:|:---:|:---:|:---:|:---:| | multilingual-e5-large | 560M | 58.81 | 61.5 | 60.16 | bge-m3 (dense) | 560M | 60.80 | 59.84 | 60.32 | gte-multilingual-base (dense) | 305M | 62.72 | 61.40 | 62.06 | KaLM-embedding-multilingual-mini-v1 | 494M | 62.31 | 61.87 | 62.09 | KaLM-embedding-multilingual-mini-instruct-v1 | 494M | 63.57 | 64.74 | 64.16 ## Requirements Since we have used the Qwen2 model, we advise you to install transformers>=4.37.0, or you might encounter the following error: KeyError: 'qwen2' ## Usage Using this model becomes easy when you have sentence-transformers installed: pip install -U sentence-transformers Then you can use the model like this: python from sentence_transformers import SentenceTransformer sentences = ["This is an example sentence", "Each sentence is converted"] model = SentenceTransformer('{MODEL_NAME_OR_PATH}') # Do NOT set trust_remote_code model.max_seq_length = 512 embeddings = model.encode( sentences, normalize_embeddings=True, batch_size=256, show_progress_bar=True ) print(embeddings) We add instruction for asymmetric tasks: retrieval, reranking, classification and clustering. If you want to add instruction to the query (no instruction for the corpus), you can use the model like this: python from sentence_transformers import SentenceTransformer sentences = ["This is an example sentence", "Each sentence is converted"] model = SentenceTransformer('{MODEL_NAME_OR_PATH}') # Do NOT set trust_remote_code model.max_seq_length = 512 prompt = "Instruct: Classifying the category of french news. \n Query: " embeddings = model.encode( sentences, prompt=prompt, normalize_embeddings=True, batch_size=256, show_progress_bar=True ) print(embeddings) ## Contact If you encounter any issue, feel free to contact us via the email: yanshek.woo@gmail.com