mirror of
https://github.com/microsoft/SkillOpt.git
synced 2026-07-07 16:49:08 +08:00
Release data split manifests
This commit is contained in:
hwq
15
.gitignore
vendored
15
.gitignore
vendored
@@ -5,7 +5,20 @@ build/
|
||||
dist/
|
||||
site/
|
||||
|
||||
data/
|
||||
data/*
|
||||
!data/README.md
|
||||
!data/searchqa_id_split/
|
||||
!data/searchqa_id_split/**
|
||||
!data/livemathematicianbench_id_split/
|
||||
!data/livemathematicianbench_id_split/**
|
||||
!data/docvqa_id_split/
|
||||
!data/docvqa_id_split/**
|
||||
!data/officeqa_id_split/
|
||||
!data/officeqa_id_split/**
|
||||
!data/spreadsheetbench_id_split/
|
||||
!data/spreadsheetbench_id_split/**
|
||||
!data/alfworld_path_split/
|
||||
!data/alfworld_path_split/**
|
||||
outputs/
|
||||
logs/
|
||||
external/
|
||||
|
||||
@@ -139,7 +139,7 @@ python scripts/train.py \
|
||||
# Train on ALFWorld:
|
||||
python scripts/train.py \
|
||||
--config configs/alfworld/default.yaml \
|
||||
--split_dir /path/to/your/alfworld_split \
|
||||
--split_dir data/alfworld_path_split \
|
||||
--azure_openai_endpoint https://your-resource.openai.azure.com/ \
|
||||
--optimizer_model gpt-5.5 \
|
||||
--target_model gpt-5.5
|
||||
|
||||
@@ -19,7 +19,7 @@ env:
|
||||
name: alfworld
|
||||
skill_init: skillopt/envs/alfworld/skills/initial.md
|
||||
split_mode: split_dir
|
||||
split_dir: data/ablation_splits/alfworld/2-1-7_seed42
|
||||
split_dir: data/alfworld_path_split
|
||||
data_path: ""
|
||||
split_output_dir: ""
|
||||
max_steps: 50
|
||||
|
||||
@@ -9,7 +9,7 @@ env:
|
||||
name: livemathematicianbench
|
||||
skill_init: skillopt/envs/livemathematicianbench/skills/initial.md
|
||||
split_mode: split_dir
|
||||
split_dir: data/ablation_splits/livemathematicianbench/2-1-7_seed42
|
||||
split_dir: data/livemathematicianbench_split
|
||||
data_path: ""
|
||||
split_output_dir: ""
|
||||
max_turns: 1
|
||||
|
||||
223
data/README.md
Normal file
223
data/README.md
Normal file
@@ -0,0 +1,223 @@
|
||||
# Data Manifests
|
||||
|
||||
This directory releases lightweight split manifests for the SkillOpt paper
|
||||
splits. These manifests are not full runnable benchmark payloads. To evaluate a
|
||||
benchmark, first materialize the full examples from the raw data source when
|
||||
needed, then point `--split_dir` at the split directory listed below.
|
||||
|
||||
In this README, "coverage" describes which part of the upstream benchmark the
|
||||
manifest references. It does not mean the released manifest directory contains
|
||||
the full runnable examples.
|
||||
|
||||
## Layout
|
||||
|
||||
Every released manifest directory uses the same file layout:
|
||||
|
||||
```text
|
||||
data/<benchmark>_<manifest_type>/
|
||||
|-- split_manifest.json
|
||||
|-- train/items.json
|
||||
|-- val/items.json
|
||||
`-- test/items.json
|
||||
```
|
||||
|
||||
`split_manifest.json` records source metadata, split counts, and item fields.
|
||||
Each `items.json` contains only stable IDs or source-path hints.
|
||||
|
||||
## Released Splits
|
||||
|
||||
| Manifest directory | Benchmark | Counts | Coverage | Raw data source | `split_dir` |
|
||||
|---|---|---:|---|---|---|
|
||||
| `searchqa_id_split/` | SearchQA | 400 / 200 / 1400 | Official HF dataset IDs | [lucadiliello/searchqa](https://huggingface.co/datasets/lucadiliello/searchqa) | `data/searchqa_split` |
|
||||
| `livemathematicianbench_id_split/` | LiveMathematicianBench | 35 / 18 / 124 | Four official monthly files | [LiveMathematicianBench/LiveMathematicianBench](https://huggingface.co/datasets/LiveMathematicianBench/LiveMathematicianBench) | `data/livemathematicianbench_split` |
|
||||
| `docvqa_id_split/` | DocVQA | 107 / 53 / 374 | 10% subset of validation | [lmms-lab/DocVQA](https://huggingface.co/datasets/lmms-lab/DocVQA) | `data/docvqa/splits` |
|
||||
| `officeqa_id_split/` | OfficeQA | 50 / 24 / 172 | OfficeQA Full | [databricks/officeqa](https://huggingface.co/datasets/databricks/officeqa) | `data/officeqa_split` |
|
||||
| `spreadsheetbench_id_split/` | SpreadsheetBench | 80 / 40 / 280 | SpreadsheetBench Verified 400 | [KAKA22/SpreadsheetBench](https://huggingface.co/datasets/KAKA22/SpreadsheetBench) | `data/spreadsheetbench_split` |
|
||||
| `alfworld_path_split/` | ALFWorld | 39 / 18 / 134 | ALFWorld `json_2.1.1` paths | [alfworld/alfworld](https://github.com/alfworld/alfworld) | `data/alfworld_path_split` |
|
||||
|
||||
Counts are ordered as train / val / test.
|
||||
|
||||
## Direct Use
|
||||
|
||||
Only `alfworld_path_split/` can be used directly as `--split_dir` from this
|
||||
release, because the ALFWorld loader reads `gamefile` and `task_type` from the
|
||||
split items.
|
||||
|
||||
This does not mean the ALFWorld raw data is included. You still need to
|
||||
download ALFWorld separately with `alfworld-download` and set `$ALFWORLD_DATA`
|
||||
to the data root containing `json_2.1.1`.
|
||||
|
||||
The other manifest directories are lookup manifests. They intentionally omit
|
||||
full example fields such as questions, answers, contexts, images, or task
|
||||
instructions. Materialize those benchmarks into the `split_dir` paths listed
|
||||
above before running SkillOpt.
|
||||
|
||||
## Lookup Keys
|
||||
|
||||
The manifests are sufficient to locate the corresponding raw examples after
|
||||
the raw data has been downloaded or otherwise made available:
|
||||
|
||||
| Benchmark | Manifest lookup key |
|
||||
|---|---|
|
||||
| SearchQA | Match `items.json[].id` to the `key` field in `lucadiliello/searchqa`. |
|
||||
| LiveMathematicianBench | Open `source_file`, then match `no`; the manifest `id` is `<month>:<no>`. |
|
||||
| DocVQA | Match `questionId` within the official DocVQA `validation` split; `image_path` records the expected local image path. |
|
||||
| OfficeQA | Match `uid` in `officeqa_full.csv`; `source_files` and `source_docs` identify the supporting document. |
|
||||
| SpreadsheetBench | Match `id`; `spreadsheet_path` identifies the referenced spreadsheet directory. |
|
||||
| ALFWorld | Resolve `gamefile` relative to `$ALFWORLD_DATA`. |
|
||||
|
||||
## Manifest Item Examples
|
||||
|
||||
SearchQA:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "221c83e6630f4e7983da48fa28da1882"
|
||||
}
|
||||
```
|
||||
|
||||
LiveMathematicianBench:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "202602:22",
|
||||
"month": "202602",
|
||||
"no": 22,
|
||||
"paper_link": "http://arxiv.org/abs/2602.10700v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
}
|
||||
```
|
||||
|
||||
DocVQA:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "50877",
|
||||
"questionId": "50877",
|
||||
"docId": "14724",
|
||||
"image_path": "data/docvqa_images/q50877_d14724.png",
|
||||
"source_split": "validation"
|
||||
}
|
||||
```
|
||||
|
||||
OfficeQA:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "UID0002",
|
||||
"uid": "UID0002",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1944_01.txt"
|
||||
}
|
||||
```
|
||||
|
||||
SpreadsheetBench:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "32438",
|
||||
"spreadsheet_path": "spreadsheet/32438",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
}
|
||||
```
|
||||
|
||||
ALFWorld:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "train:0000",
|
||||
"gamefile": "json_2.1.1/train/.../game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
}
|
||||
```
|
||||
|
||||
## Benchmark Notes
|
||||
|
||||
### SearchQA
|
||||
|
||||
`searchqa_id_split/` is an ID-only manifest. Each released `id` exactly matches
|
||||
the `key` field in `lucadiliello/searchqa`.
|
||||
|
||||
Materialized examples must include the fields consumed by the SearchQA
|
||||
environment, including:
|
||||
|
||||
```text
|
||||
question
|
||||
context
|
||||
answers
|
||||
```
|
||||
|
||||
### LiveMathematicianBench
|
||||
|
||||
`livemathematicianbench_id_split/` was generated from these raw files:
|
||||
|
||||
```text
|
||||
data/202511/qa_202511_final.json
|
||||
data/202512/qa_202512_final.json
|
||||
data/202601/qa_202601_final.json
|
||||
data/202602/qa_202602_final.json
|
||||
```
|
||||
|
||||
The manifest stores IDs in the loader format:
|
||||
|
||||
```text
|
||||
<month>:<no>
|
||||
```
|
||||
|
||||
Materialized examples must include:
|
||||
|
||||
```text
|
||||
question
|
||||
choices
|
||||
correct_choice
|
||||
theorem_type
|
||||
theorem
|
||||
sketch
|
||||
paper_link
|
||||
```
|
||||
|
||||
### DocVQA
|
||||
|
||||
`docvqa_id_split/` records `docvqa_validation_10pct`: a 10% subset sampled from
|
||||
the official DocVQA `validation` split.
|
||||
|
||||
```text
|
||||
source_split: validation
|
||||
docvqa_validation_10pct: train=107, val=53, test=374
|
||||
```
|
||||
|
||||
Each manifest item contains question/document IDs plus image location metadata.
|
||||
Materialized examples must provide `question`, `answer` or `ground_truth`, and
|
||||
an `image_path` that resolves locally.
|
||||
|
||||
### OfficeQA
|
||||
|
||||
`officeqa_id_split/` records the split over OfficeQA Full
|
||||
(`officeqa_full.csv`). The official OfficeQA CSVs are gated on Hugging Face, so
|
||||
materialization requires authorized access.
|
||||
|
||||
Each manifest item contains `uid`, `category`, `source_files`, and
|
||||
`source_docs` hints. Materialized examples must include `question` and
|
||||
`ground_truth` or `answer`.
|
||||
|
||||
### SpreadsheetBench
|
||||
|
||||
`spreadsheetbench_id_split/` records the split over SpreadsheetBench Verified
|
||||
400, from `spreadsheetbench_verified_400.tar.gz`.
|
||||
|
||||
Each manifest item contains task identity metadata such as `id`,
|
||||
`spreadsheet_path`, and `instruction_type`. Materialization must also place the
|
||||
referenced spreadsheet directories at:
|
||||
|
||||
```text
|
||||
data/spreadsheetbench_verified_400
|
||||
```
|
||||
|
||||
### ALFWorld
|
||||
|
||||
`alfworld_path_split/` records `gamefile` paths relative to `$ALFWORLD_DATA`.
|
||||
The source payload is `json_2.1.1`, which must be downloaded separately with
|
||||
`alfworld-download`.
|
||||
|
||||
This manifest can be used directly as `--split_dir` after `$ALFWORLD_DATA`
|
||||
points to the local ALFWorld data root containing `json_2.1.1`.
|
||||
29
data/alfworld_path_split/split_manifest.json
Normal file
29
data/alfworld_path_split/split_manifest.json
Normal file
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"benchmark": "ALFWorld",
|
||||
"manifest_type": "path_split",
|
||||
"source_repo": "alfworld/alfworld",
|
||||
"source_repo_type": "repository",
|
||||
"source_url": "https://github.com/alfworld/alfworld",
|
||||
"source_file": "json_2.1.1",
|
||||
"source_method": "generated by alfworld-download",
|
||||
"source_split_files": [
|
||||
"split_train.json",
|
||||
"split_val.json",
|
||||
"split_test.json"
|
||||
],
|
||||
"counts": {
|
||||
"train": 39,
|
||||
"val": 18,
|
||||
"test": 134
|
||||
},
|
||||
"item_fields": [
|
||||
"id",
|
||||
"gamefile",
|
||||
"task_type"
|
||||
],
|
||||
"path_root": "$ALFWORLD_DATA",
|
||||
"notes": [
|
||||
"This is a path manifest, not the ALFWorld game payload.",
|
||||
"The gamefile field is relative to ALFWORLD_DATA and must be expanded before direct use as split_dir data."
|
||||
]
|
||||
}
|
||||
672
data/alfworld_path_split/test/items.json
Normal file
672
data/alfworld_path_split/test/items.json
Normal file
@@ -0,0 +1,672 @@
|
||||
[
|
||||
{
|
||||
"id": "test:0000",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-AlarmClock-None-DeskLamp-308/trial_T20190908_222917_366542/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0001",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-AlarmClock-None-DeskLamp-308/trial_T20190908_222933_607649/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0002",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-AlarmClock-None-DeskLamp-308/trial_T20190908_222951_616606/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0003",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-Book-None-DeskLamp-308/trial_T20190908_020029_636862/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0004",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-Book-None-DeskLamp-308/trial_T20190908_020048_814402/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0005",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-Book-None-DeskLamp-308/trial_T20190908_144951_587345/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0006",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-Bowl-None-DeskLamp-308/trial_T20190907_133919_856963/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0007",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-Bowl-None-DeskLamp-308/trial_T20190907_133935_066606/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0008",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-Bowl-None-DeskLamp-308/trial_T20190907_133953_562557/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0009",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-CD-None-DeskLamp-308/trial_T20190908_141942_810052/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0010",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-CD-None-DeskLamp-308/trial_T20190908_141958_463362/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0011",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-CD-None-DeskLamp-308/trial_T20190908_142046_281296/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0012",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-Mug-None-DeskLamp-308/trial_T20190908_161733_213242/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0013",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-Mug-None-DeskLamp-308/trial_T20190908_201421_021646/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0014",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-Mug-None-DeskLamp-308/trial_T20190908_201444_037645/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0015",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-Pencil-None-DeskLamp-308/trial_T20190908_220545_153480/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0016",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-Pencil-None-DeskLamp-308/trial_T20190908_220604_010430/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0017",
|
||||
"gamefile": "json_2.1.1/valid_unseen/look_at_obj_in_light-Pencil-None-DeskLamp-308/trial_T20190908_220656_510400/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "test:0018",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-Mug-None-Desk-308/trial_T20190908_125200_737896/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0019",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-Mug-None-Desk-308/trial_T20190909_203041_433487/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0020",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-Mug-None-Desk-308/trial_T20190909_210238_431966/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0021",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-Pencil-None-Shelf-308/trial_T20190908_121952_610012/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0022",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-Pencil-None-Shelf-308/trial_T20190908_122024_052056/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0023",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-Pencil-None-Shelf-308/trial_T20190908_122154_042763/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0024",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-PepperShaker-None-Drawer-10/trial_T20190906_184021_215264/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0025",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-PepperShaker-None-Drawer-10/trial_T20190918_154326_823501/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0026",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-PepperShaker-None-Drawer-10/trial_T20190918_154424_844749/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0027",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-SaltShaker-None-Cabinet-10/trial_T20190906_191429_743650/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0028",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-SaltShaker-None-Cabinet-10/trial_T20190906_191445_723170/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0029",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-SaltShaker-None-Cabinet-10/trial_T20190906_191501_563086/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0030",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-SaltShaker-None-Drawer-10/trial_T20190909_021613_077537/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0031",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-SaltShaker-None-Drawer-10/trial_T20190909_021650_880235/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0032",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-SaltShaker-None-Drawer-10/trial_T20190909_021728_339782/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0033",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-SoapBottle-None-Toilet-424/trial_T20190907_004321_405868/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0034",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-SoapBottle-None-Toilet-424/trial_T20190907_004351_281384/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0035",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-SoapBottle-None-Toilet-424/trial_T20190907_004404_604165/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0036",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-Vase-None-Safe-219/trial_T20190908_205204_244321/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0037",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-Vase-None-Safe-219/trial_T20190908_205221_748352/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0038",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-Vase-None-Safe-219/trial_T20190908_205246_776817/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0039",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-Watch-None-Safe-219/trial_T20190907_074524_006355/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0040",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-Watch-None-Safe-219/trial_T20190907_074556_124850/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0041",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_and_place_simple-Watch-None-Safe-219/trial_T20190907_074643_810052/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "test:0042",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Bowl-None-Cabinet-10/trial_T20190909_061130_844814/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0043",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Bowl-None-Cabinet-10/trial_T20190909_061158_110530/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0044",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Bowl-None-Cabinet-10/trial_T20190909_061232_368489/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0045",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Cloth-None-Cabinet-424/trial_T20190908_022321_380927/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0046",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Cloth-None-Cabinet-424/trial_T20190908_022436_073995/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0047",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Cloth-None-CounterTop-424/trial_T20190908_100632_546757/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0048",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Cloth-None-CounterTop-424/trial_T20190908_114340_674467/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0049",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Egg-None-Microwave-10/trial_T20190909_120554_888709/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0050",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Egg-None-Microwave-10/trial_T20190909_120632_691361/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0051",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Egg-None-Microwave-10/trial_T20190909_120712_273910/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0052",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Knife-None-CounterTop-10/trial_T20190909_110347_624008/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0053",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Knife-None-CounterTop-10/trial_T20190909_110445_675754/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0054",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Knife-None-CounterTop-10/trial_T20190909_110531_148235/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0055",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Mug-None-CoffeeMachine-10/trial_T20190907_221208_560499/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0056",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Mug-None-CoffeeMachine-10/trial_T20190907_221300_362511/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0057",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Mug-None-CoffeeMachine-10/trial_T20190907_221355_558505/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0058",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Pan-None-CounterTop-10/trial_T20190908_032434_013084/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0059",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Pan-None-CounterTop-10/trial_T20190908_032518_891433/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0060",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Pan-None-CounterTop-10/trial_T20190908_032543_712058/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0061",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Plate-None-CounterTop-10/trial_T20190908_213356_017769/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0062",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Plate-None-CounterTop-10/trial_T20190908_213420_728917/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0063",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Plate-None-CounterTop-10/trial_T20190908_213533_897289/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0064",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-SoapBar-None-Cabinet-424/trial_T20190908_214926_337906/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0065",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-SoapBar-None-Cabinet-424/trial_T20190908_214946_567644/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0066",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-SoapBar-None-Cabinet-424/trial_T20190908_215019_162873/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0067",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-SoapBar-None-CounterTop-424/trial_T20190907_074045_109439/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0068",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-SoapBar-None-CounterTop-424/trial_T20190907_074106_050405/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0069",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-SoapBar-None-CounterTop-424/trial_T20190907_074124_966890/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0070",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Spatula-None-Drawer-10/trial_T20190907_080730_211959/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0071",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Spatula-None-Drawer-10/trial_T20190907_080800_275989/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0072",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_clean_then_place_in_recep-Spatula-None-Drawer-10/trial_T20190907_080825_222432/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0073",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Bread-None-CounterTop-10/trial_T20190908_091747_866951/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0074",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Bread-None-CounterTop-10/trial_T20190908_091811_414150/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0075",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Bread-None-CounterTop-10/trial_T20190908_091835_825830/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0076",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Lettuce-None-CounterTop-10/trial_T20190909_123133_763972/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0077",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Lettuce-None-CounterTop-10/trial_T20190909_174807_646433/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0078",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Lettuce-None-CounterTop-10/trial_T20190909_174840_771703/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0079",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Mug-None-Cabinet-10/trial_T20190909_121559_082363/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0080",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Mug-None-Cabinet-10/trial_T20190909_121635_622676/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0081",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Mug-None-Cabinet-10/trial_T20190909_121710_650938/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0082",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Mug-None-CoffeeMachine-10/trial_T20190907_183715_299073/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0083",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Mug-None-CoffeeMachine-10/trial_T20190907_183807_477267/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0084",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Mug-None-CoffeeMachine-10/trial_T20190907_183853_958104/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0085",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Pan-None-CounterTop-10/trial_T20190908_114545_244903/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0086",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Pan-None-CounterTop-10/trial_T20190908_114622_738670/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0087",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Pan-None-CounterTop-10/trial_T20190908_114656_768805/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0088",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Potato-None-Microwave-10/trial_T20190907_033157_424297/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0089",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Potato-None-Microwave-10/trial_T20190907_033228_194678/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0090",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Potato-None-Microwave-10/trial_T20190907_033306_962974/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0091",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Tomato-None-Microwave-10/trial_T20190909_102608_318800/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0092",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Tomato-None-Microwave-10/trial_T20190909_102644_926781/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0093",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_cool_then_place_in_recep-Tomato-None-Microwave-10/trial_T20190909_102710_795182/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0094",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Apple-None-Fridge-10/trial_T20190906_182259_116320/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0095",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Apple-None-Fridge-10/trial_T20190906_182353_418140/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0096",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Apple-None-Fridge-10/trial_T20190906_182435_622538/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0097",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Apple-None-GarbageCan-10/trial_T20190908_145050_918567/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0098",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Apple-None-GarbageCan-10/trial_T20190908_145143_820541/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0099",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Apple-None-GarbageCan-10/trial_T20190908_145356_918528/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0100",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Cup-None-Cabinet-10/trial_T20190907_083346_800823/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0101",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Cup-None-Cabinet-10/trial_T20190907_083429_887065/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0102",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Cup-None-Cabinet-10/trial_T20190907_083507_594820/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0103",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Egg-None-GarbageCan-10/trial_T20190908_113432_673307/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0104",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Egg-None-GarbageCan-10/trial_T20190908_113523_123938/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0105",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Egg-None-GarbageCan-10/trial_T20190908_113610_425142/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0106",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Mug-None-Cabinet-10/trial_T20190909_021100_341887/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0107",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Mug-None-Cabinet-10/trial_T20190909_021200_669381/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0108",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Mug-None-Cabinet-10/trial_T20190909_021247_306737/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0109",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Mug-None-CoffeeMachine-10/trial_T20190907_171806_406231/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0110",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Mug-None-CoffeeMachine-10/trial_T20190907_171850_960211/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0111",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Mug-None-CoffeeMachine-10/trial_T20190907_171933_349922/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0112",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Potato-None-GarbageCan-10/trial_T20190907_161745_664033/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0113",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Potato-None-GarbageCan-10/trial_T20190907_161853_945788/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0114",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Tomato-None-GarbageCan-10/trial_T20190908_225046_020282/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0115",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Tomato-None-GarbageCan-10/trial_T20190908_225359_617900/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0116",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_heat_then_place_in_recep-Tomato-None-GarbageCan-10/trial_T20190908_225453_272533/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "test:0117",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-CD-None-Safe-308/trial_T20190907_050942_897916/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0118",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-CD-None-Safe-308/trial_T20190907_051013_060265/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0119",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-CD-None-Safe-308/trial_T20190907_051056_585414/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0120",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-KeyChain-None-Safe-219/trial_T20190909_011803_423115/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0121",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-KeyChain-None-Safe-219/trial_T20190909_012027_782483/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0122",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-PepperShaker-None-Drawer-10/trial_T20190908_010306_215435/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0123",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-PepperShaker-None-Drawer-10/trial_T20190912_221016_460197/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0124",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-PepperShaker-None-Drawer-10/trial_T20190912_221141_608117/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0125",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-Pillow-None-Sofa-219/trial_T20190907_163240_345855/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0126",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-Pillow-None-Sofa-219/trial_T20190907_163327_486300/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0127",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-Pillow-None-Sofa-219/trial_T20190907_163408_914117/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0128",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-SoapBar-None-Cabinet-424/trial_T20190909_081720_491733/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0129",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-SoapBar-None-Cabinet-424/trial_T20190909_081746_857594/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0130",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-SoapBar-None-GarbageCan-424/trial_T20190909_064053_839817/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0131",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-SoapBar-None-GarbageCan-424/trial_T20190909_064221_368939/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0132",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-SoapBar-None-GarbageCan-424/trial_T20190909_064309_357168/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "test:0133",
|
||||
"gamefile": "json_2.1.1/valid_unseen/pick_two_obj_and_place-ToiletPaper-None-Cabinet-424/trial_T20190906_202926_527010/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
}
|
||||
]
|
||||
197
data/alfworld_path_split/train/items.json
Normal file
197
data/alfworld_path_split/train/items.json
Normal file
@@ -0,0 +1,197 @@
|
||||
[
|
||||
{
|
||||
"id": "train:0000",
|
||||
"gamefile": "json_2.1.1/train/look_at_obj_in_light-AlarmClock-None-DeskLamp-305/trial_T20190908_082736_108723/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "train:0001",
|
||||
"gamefile": "json_2.1.1/train/look_at_obj_in_light-CD-None-DeskLamp-304/trial_T20190907_185649_782438/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "train:0002",
|
||||
"gamefile": "json_2.1.1/train/look_at_obj_in_light-CD-None-DeskLamp-320/trial_T20190907_224439_174735/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "train:0003",
|
||||
"gamefile": "json_2.1.1/train/look_at_obj_in_light-Pillow-None-DeskLamp-316/trial_T20190908_232421_645610/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "train:0004",
|
||||
"gamefile": "json_2.1.1/train/look_at_obj_in_light-Statue-None-DeskLamp-319/trial_T20190907_035546_167548/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "train:0005",
|
||||
"gamefile": "json_2.1.1/train/pick_and_place_simple-CellPhone-None-Shelf-313/trial_T20190908_123725_452958/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "train:0006",
|
||||
"gamefile": "json_2.1.1/train/pick_and_place_simple-Newspaper-None-Sofa-211/trial_T20190906_175004_203092/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "train:0007",
|
||||
"gamefile": "json_2.1.1/train/pick_and_place_simple-Pencil-None-Desk-302/trial_T20190908_032836_462632/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "train:0008",
|
||||
"gamefile": "json_2.1.1/train/pick_and_place_simple-SoapBar-None-GarbageCan-416/trial_T20190908_020839_714699/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "train:0009",
|
||||
"gamefile": "json_2.1.1/train/pick_and_place_simple-Statue-None-CoffeeTable-222/trial_T20190907_131249_788749/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "train:0010",
|
||||
"gamefile": "json_2.1.1/train/pick_and_place_simple-ToiletPaper-None-ToiletPaperHanger-406/trial_T20190908_122807_136741/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "train:0011",
|
||||
"gamefile": "json_2.1.1/train/pick_and_place_simple-ToiletPaper-None-ToiletPaperHanger-415/trial_T20190908_050443_333939/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "train:0012",
|
||||
"gamefile": "json_2.1.1/train/pick_clean_then_place_in_recep-Apple-None-DiningTable-4/trial_T20190908_104413_450768/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0013",
|
||||
"gamefile": "json_2.1.1/train/pick_clean_then_place_in_recep-DishSponge-None-Shelf-20/trial_T20190907_222429_992578/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0014",
|
||||
"gamefile": "json_2.1.1/train/pick_clean_then_place_in_recep-DishSponge-None-Shelf-401/trial_T20190908_072225_397518/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0015",
|
||||
"gamefile": "json_2.1.1/train/pick_clean_then_place_in_recep-Kettle-None-Cabinet-2/trial_T20190909_043103_418752/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0016",
|
||||
"gamefile": "json_2.1.1/train/pick_clean_then_place_in_recep-Knife-None-Drawer-22/trial_T20190907_224827_746945/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0017",
|
||||
"gamefile": "json_2.1.1/train/pick_clean_then_place_in_recep-Lettuce-None-DiningTable-20/trial_T20190906_191148_519826/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0018",
|
||||
"gamefile": "json_2.1.1/train/pick_clean_then_place_in_recep-Lettuce-None-Fridge-13/trial_T20190908_203022_601787/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0019",
|
||||
"gamefile": "json_2.1.1/train/pick_clean_then_place_in_recep-Plate-None-Fridge-5/trial_T20190909_112954_869911/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0020",
|
||||
"gamefile": "json_2.1.1/train/pick_clean_then_place_in_recep-Spoon-None-DiningTable-18/trial_T20190909_102159_277894/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0021",
|
||||
"gamefile": "json_2.1.1/train/pick_cool_then_place_in_recep-Bread-None-CounterTop-1/trial_T20190908_212439_711334/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0022",
|
||||
"gamefile": "json_2.1.1/train/pick_cool_then_place_in_recep-Bread-None-CounterTop-15/trial_T20190909_085448_256298/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0023",
|
||||
"gamefile": "json_2.1.1/train/pick_cool_then_place_in_recep-Bread-None-CounterTop-16/trial_T20190908_143948_082471/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0024",
|
||||
"gamefile": "json_2.1.1/train/pick_cool_then_place_in_recep-Pan-None-StoveBurner-27/trial_T20190906_212619_469871/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0025",
|
||||
"gamefile": "json_2.1.1/train/pick_cool_then_place_in_recep-Plate-None-DiningTable-17/trial_T20190909_122939_032098/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0026",
|
||||
"gamefile": "json_2.1.1/train/pick_cool_then_place_in_recep-Pot-None-CounterTop-1/trial_T20190909_124252_504581/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0027",
|
||||
"gamefile": "json_2.1.1/train/pick_heat_then_place_in_recep-Apple-None-Fridge-20/trial_T20190908_013911_274341/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0028",
|
||||
"gamefile": "json_2.1.1/train/pick_heat_then_place_in_recep-Egg-None-CounterTop-12/trial_T20190908_215527_416490/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0029",
|
||||
"gamefile": "json_2.1.1/train/pick_heat_then_place_in_recep-Mug-None-CoffeeMachine-1/trial_T20190907_222924_821086/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0030",
|
||||
"gamefile": "json_2.1.1/train/pick_heat_then_place_in_recep-Mug-None-CoffeeMachine-28/trial_T20190908_062730_537428/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0031",
|
||||
"gamefile": "json_2.1.1/train/pick_heat_then_place_in_recep-Plate-None-Cabinet-13/trial_T20190907_062749_759882/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0032",
|
||||
"gamefile": "json_2.1.1/train/pick_heat_then_place_in_recep-Potato-None-Fridge-2/trial_T20190909_030845_198194/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0033",
|
||||
"gamefile": "json_2.1.1/train/pick_heat_then_place_in_recep-Tomato-None-CounterTop-26/trial_T20190907_005525_499114/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "train:0034",
|
||||
"gamefile": "json_2.1.1/train/pick_two_obj_and_place-CD-None-Drawer-319/trial_T20190907_145515_348252/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "train:0035",
|
||||
"gamefile": "json_2.1.1/train/pick_two_obj_and_place-Candle-None-Drawer-427/trial_T20190909_043917_251333/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "train:0036",
|
||||
"gamefile": "json_2.1.1/train/pick_two_obj_and_place-KeyChain-None-ArmChair-222/trial_T20190909_100312_677332/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "train:0037",
|
||||
"gamefile": "json_2.1.1/train/pick_two_obj_and_place-Newspaper-None-Sofa-212/trial_T20190908_112632_208041/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "train:0038",
|
||||
"gamefile": "json_2.1.1/train/pick_two_obj_and_place-SaltShaker-None-SideTable-21/trial_T20190909_041626_844806/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
}
|
||||
]
|
||||
92
data/alfworld_path_split/val/items.json
Normal file
92
data/alfworld_path_split/val/items.json
Normal file
@@ -0,0 +1,92 @@
|
||||
[
|
||||
{
|
||||
"id": "val:0000",
|
||||
"gamefile": "json_2.1.1/valid_seen/look_at_obj_in_light-AlarmClock-None-DeskLamp-323/trial_T20190909_044715_250790/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "val:0001",
|
||||
"gamefile": "json_2.1.1/valid_seen/look_at_obj_in_light-Bowl-None-DeskLamp-301/trial_T20190909_150719_492274/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "val:0002",
|
||||
"gamefile": "json_2.1.1/valid_seen/look_at_obj_in_light-Pillow-None-DeskLamp-323/trial_T20190908_053153_077977/game.tw-pddl",
|
||||
"task_type": "look_at_obj_in_light"
|
||||
},
|
||||
{
|
||||
"id": "val:0003",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_and_place_simple-Mug-None-SideTable-329/trial_T20190909_032318_169393/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "val:0004",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_and_place_simple-Mug-None-SideTable-329/trial_T20190909_032340_274147/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "val:0005",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_and_place_simple-Pencil-None-Desk-310/trial_T20190909_113054_894334/game.tw-pddl",
|
||||
"task_type": "pick_and_place_simple"
|
||||
},
|
||||
{
|
||||
"id": "val:0006",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_clean_then_place_in_recep-ButterKnife-None-Drawer-30/trial_T20190908_052007_212776/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "val:0007",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_clean_then_place_in_recep-ButterKnife-None-Drawer-8/trial_T20190909_124425_112757/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "val:0008",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_clean_then_place_in_recep-SoapBar-None-Cabinet-402/trial_T20190908_055221_984342/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "val:0009",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_clean_then_place_in_recep-SoapBar-None-Toilet-410/trial_T20190906_201106_979461/game.tw-pddl",
|
||||
"task_type": "pick_clean_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "val:0010",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_cool_then_place_in_recep-Apple-None-Microwave-19/trial_T20190906_210937_878489/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "val:0011",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_cool_then_place_in_recep-Plate-None-CounterTop-1/trial_T20190906_205324_559361/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "val:0012",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_cool_then_place_in_recep-Tomato-None-Microwave-18/trial_T20190909_012524_159092/game.tw-pddl",
|
||||
"task_type": "pick_cool_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "val:0013",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_heat_then_place_in_recep-Apple-None-DiningTable-26/trial_T20190907_060234_011675/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "val:0014",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_heat_then_place_in_recep-Tomato-None-Fridge-15/trial_T20190909_020200_054379/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "val:0015",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_heat_then_place_in_recep-Tomato-None-Fridge-23/trial_T20190909_082320_103350/game.tw-pddl",
|
||||
"task_type": "pick_heat_then_place_in_recep"
|
||||
},
|
||||
{
|
||||
"id": "val:0016",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_two_obj_and_place-Book-None-Desk-313/trial_T20190908_125930_920681/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
},
|
||||
{
|
||||
"id": "val:0017",
|
||||
"gamefile": "json_2.1.1/valid_seen/pick_two_obj_and_place-CreditCard-None-Safe-323/trial_T20190907_001129_214240/game.tw-pddl",
|
||||
"task_type": "pick_two_obj_and_place"
|
||||
}
|
||||
]
|
||||
36
data/docvqa_id_split/split_manifest.json
Normal file
36
data/docvqa_id_split/split_manifest.json
Normal file
@@ -0,0 +1,36 @@
|
||||
{
|
||||
"benchmark": "DocVQA",
|
||||
"manifest_type": "id_split",
|
||||
"source_repo": "lmms-lab/DocVQA",
|
||||
"source_repo_type": "dataset",
|
||||
"source_url": "https://huggingface.co/datasets/lmms-lab/DocVQA",
|
||||
"source_revision": "539088ef8a8ada01ac8e2e6d4e372586748a265e",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"source_split_name": "docvqa_validation_10pct",
|
||||
"split_method": "10% subset sampled from the DocVQA validation split",
|
||||
"counts": {
|
||||
"train": 107,
|
||||
"val": 53,
|
||||
"test": 374
|
||||
},
|
||||
"item_fields": [
|
||||
"id",
|
||||
"questionId",
|
||||
"docId",
|
||||
"image_path",
|
||||
"ucsf_document_id",
|
||||
"ucsf_document_page_no",
|
||||
"topic",
|
||||
"source_dataset",
|
||||
"source_config",
|
||||
"source_split",
|
||||
"sample_seed"
|
||||
],
|
||||
"notes": [
|
||||
"This is a split manifest, not the full DocVQA payload.",
|
||||
"Materialize full CSV rows and image files before evaluation.",
|
||||
"This manifest corresponds to docvqa_validation_10pct.",
|
||||
"All released train/val/test items originate from a 10% subset of the official DocVQA validation split."
|
||||
]
|
||||
}
|
||||
4864
data/docvqa_id_split/test/items.json
Normal file
4864
data/docvqa_id_split/test/items.json
Normal file
File diff suppressed because it is too large
Load Diff
1393
data/docvqa_id_split/train/items.json
Normal file
1393
data/docvqa_id_split/train/items.json
Normal file
File diff suppressed because it is too large
Load Diff
691
data/docvqa_id_split/val/items.json
Normal file
691
data/docvqa_id_split/val/items.json
Normal file
@@ -0,0 +1,691 @@
|
||||
[
|
||||
{
|
||||
"id": "62409",
|
||||
"questionId": "62409",
|
||||
"docId": "8554",
|
||||
"image_path": "data/docvqa_images/q62409_d8554.png",
|
||||
"ucsf_document_id": "pgjw0227",
|
||||
"ucsf_document_page_no": "5",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "50961",
|
||||
"questionId": "50961",
|
||||
"docId": "549",
|
||||
"image_path": "data/docvqa_images/q50961_d549.png",
|
||||
"ucsf_document_id": "qtjf0226",
|
||||
"ucsf_document_page_no": "2",
|
||||
"topic": "free_text",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "46461",
|
||||
"questionId": "46461",
|
||||
"docId": "13361",
|
||||
"image_path": "data/docvqa_images/q46461_d13361.png",
|
||||
"ucsf_document_id": "ysbw0217",
|
||||
"ucsf_document_page_no": "5",
|
||||
"topic": "layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "3041",
|
||||
"questionId": "3041",
|
||||
"docId": "1204",
|
||||
"image_path": "data/docvqa_images/q3041_d1204.png",
|
||||
"ucsf_document_id": "xfjv0228",
|
||||
"ucsf_document_page_no": "3",
|
||||
"topic": "form",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "41716",
|
||||
"questionId": "41716",
|
||||
"docId": "11835",
|
||||
"image_path": "data/docvqa_images/q41716_d11835.png",
|
||||
"ucsf_document_id": "qjgn0226",
|
||||
"ucsf_document_page_no": "131",
|
||||
"topic": "form",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "61123",
|
||||
"questionId": "61123",
|
||||
"docId": "7374",
|
||||
"image_path": "data/docvqa_images/q61123_d7374.png",
|
||||
"ucsf_document_id": "mldg0227",
|
||||
"ucsf_document_page_no": "5",
|
||||
"topic": "layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "43068",
|
||||
"questionId": "43068",
|
||||
"docId": "12393",
|
||||
"image_path": "data/docvqa_images/q43068_d12393.png",
|
||||
"ucsf_document_id": "rmwn0226",
|
||||
"ucsf_document_page_no": "52",
|
||||
"topic": "figure/diagram",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "51221",
|
||||
"questionId": "51221",
|
||||
"docId": "764",
|
||||
"image_path": "data/docvqa_images/q51221_d764.png",
|
||||
"ucsf_document_id": "kzbn0226",
|
||||
"ucsf_document_page_no": "14",
|
||||
"topic": "layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "6397",
|
||||
"questionId": "6397",
|
||||
"docId": "2242",
|
||||
"image_path": "data/docvqa_images/q6397_d2242.png",
|
||||
"ucsf_document_id": "jkcn0000",
|
||||
"ucsf_document_page_no": "2",
|
||||
"topic": "form",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "57428",
|
||||
"questionId": "57428",
|
||||
"docId": "4779",
|
||||
"image_path": "data/docvqa_images/q57428_d4779.png",
|
||||
"ucsf_document_id": "rnbx0223",
|
||||
"ucsf_document_page_no": "208",
|
||||
"topic": "Image/Photo",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "3135",
|
||||
"questionId": "3135",
|
||||
"docId": "1221",
|
||||
"image_path": "data/docvqa_images/q3135_d1221.png",
|
||||
"ucsf_document_id": "ngph0227",
|
||||
"ucsf_document_page_no": "5",
|
||||
"topic": "layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "18819",
|
||||
"questionId": "18819",
|
||||
"docId": "5749",
|
||||
"image_path": "data/docvqa_images/q18819_d5749.png",
|
||||
"ucsf_document_id": "jhfd0079",
|
||||
"ucsf_document_page_no": "9",
|
||||
"topic": "form",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "15382",
|
||||
"questionId": "15382",
|
||||
"docId": "4890",
|
||||
"image_path": "data/docvqa_images/q15382_d4890.png",
|
||||
"ucsf_document_id": "kjvw0217",
|
||||
"ucsf_document_page_no": "3",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "5772",
|
||||
"questionId": "5772",
|
||||
"docId": "1940",
|
||||
"image_path": "data/docvqa_images/q5772_d1940.png",
|
||||
"ucsf_document_id": "pzyw0224",
|
||||
"ucsf_document_page_no": "10",
|
||||
"topic": "form",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "49077",
|
||||
"questionId": "49077",
|
||||
"docId": "14179",
|
||||
"image_path": "data/docvqa_images/q49077_d14179.png",
|
||||
"ucsf_document_id": "nrxb0228",
|
||||
"ucsf_document_page_no": "3",
|
||||
"topic": "layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "58519",
|
||||
"questionId": "58519",
|
||||
"docId": "5347",
|
||||
"image_path": "data/docvqa_images/q58519_d5347.png",
|
||||
"ucsf_document_id": "sjbw0217",
|
||||
"ucsf_document_page_no": "11",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "50720",
|
||||
"questionId": "50720",
|
||||
"docId": "281",
|
||||
"image_path": "data/docvqa_images/q50720_d281.png",
|
||||
"ucsf_document_id": "nrcj0037",
|
||||
"ucsf_document_page_no": "7",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "56785",
|
||||
"questionId": "56785",
|
||||
"docId": "14289",
|
||||
"image_path": "data/docvqa_images/q56785_d14289.png",
|
||||
"ucsf_document_id": "xkbv0228",
|
||||
"ucsf_document_page_no": "1",
|
||||
"topic": "layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "59653",
|
||||
"questionId": "59653",
|
||||
"docId": "6579",
|
||||
"image_path": "data/docvqa_images/q59653_d6579.png",
|
||||
"ucsf_document_id": "mzbx0227",
|
||||
"ucsf_document_page_no": "2",
|
||||
"topic": "layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "61791",
|
||||
"questionId": "61791",
|
||||
"docId": "8072",
|
||||
"image_path": "data/docvqa_images/q61791_d8072.png",
|
||||
"ucsf_document_id": "hfmf0227",
|
||||
"ucsf_document_page_no": "1",
|
||||
"topic": "layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "37229",
|
||||
"questionId": "37229",
|
||||
"docId": "10742",
|
||||
"image_path": "data/docvqa_images/q37229_d10742.png",
|
||||
"ucsf_document_id": "nkcd0227",
|
||||
"ucsf_document_page_no": "2",
|
||||
"topic": "free_text",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "60407",
|
||||
"questionId": "60407",
|
||||
"docId": "7135",
|
||||
"image_path": "data/docvqa_images/q60407_d7135.png",
|
||||
"ucsf_document_id": "gkpk0226",
|
||||
"ucsf_document_page_no": "1",
|
||||
"topic": "free_text",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "64420",
|
||||
"questionId": "64420",
|
||||
"docId": "10230",
|
||||
"image_path": "data/docvqa_images/q64420_d10230.png",
|
||||
"ucsf_document_id": "jnjm0223",
|
||||
"ucsf_document_page_no": "107",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "47365",
|
||||
"questionId": "47365",
|
||||
"docId": "13813",
|
||||
"image_path": "data/docvqa_images/q47365_d13813.png",
|
||||
"ucsf_document_id": "nxym0227",
|
||||
"ucsf_document_page_no": "28",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "47458",
|
||||
"questionId": "47458",
|
||||
"docId": "13639",
|
||||
"image_path": "data/docvqa_images/q47458_d13639.png",
|
||||
"ucsf_document_id": "skdv0228",
|
||||
"ucsf_document_page_no": "5",
|
||||
"topic": "layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "7621",
|
||||
"questionId": "7621",
|
||||
"docId": "2668",
|
||||
"image_path": "data/docvqa_images/q7621_d2668.png",
|
||||
"ucsf_document_id": "flxn0020",
|
||||
"ucsf_document_page_no": "1",
|
||||
"topic": "form",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "53575",
|
||||
"questionId": "53575",
|
||||
"docId": "2766",
|
||||
"image_path": "data/docvqa_images/q53575_d2766.png",
|
||||
"ucsf_document_id": "hsfn0020",
|
||||
"ucsf_document_page_no": "2",
|
||||
"topic": "free_text|table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "60913",
|
||||
"questionId": "60913",
|
||||
"docId": "7349",
|
||||
"image_path": "data/docvqa_images/q60913_d7349.png",
|
||||
"ucsf_document_id": "jzhd0227",
|
||||
"ucsf_document_page_no": "61",
|
||||
"topic": "free_text",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "60454",
|
||||
"questionId": "60454",
|
||||
"docId": "7163",
|
||||
"image_path": "data/docvqa_images/q60454_d7163.png",
|
||||
"ucsf_document_id": "jgyk0226",
|
||||
"ucsf_document_page_no": "1",
|
||||
"topic": "free_text",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "57978",
|
||||
"questionId": "57978",
|
||||
"docId": "4920",
|
||||
"image_path": "data/docvqa_images/q57978_d4920.png",
|
||||
"ucsf_document_id": "lkvw0217",
|
||||
"ucsf_document_page_no": "2",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "64547",
|
||||
"questionId": "64547",
|
||||
"docId": "10361",
|
||||
"image_path": "data/docvqa_images/q64547_d10361.png",
|
||||
"ucsf_document_id": "lpdl0226",
|
||||
"ucsf_document_page_no": "32",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "59481",
|
||||
"questionId": "59481",
|
||||
"docId": "6243",
|
||||
"image_path": "data/docvqa_images/q59481_d6243.png",
|
||||
"ucsf_document_id": "psgv0228",
|
||||
"ucsf_document_page_no": "5",
|
||||
"topic": "free_text",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "61472",
|
||||
"questionId": "61472",
|
||||
"docId": "7757",
|
||||
"image_path": "data/docvqa_images/q61472_d7757.png",
|
||||
"ucsf_document_id": "ymkp0227",
|
||||
"ucsf_document_page_no": "13",
|
||||
"topic": "handwritten|table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "5673",
|
||||
"questionId": "5673",
|
||||
"docId": "1908",
|
||||
"image_path": "data/docvqa_images/q5673_d1908.png",
|
||||
"ucsf_document_id": "lldj0224",
|
||||
"ucsf_document_page_no": "2",
|
||||
"topic": "form",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "49109",
|
||||
"questionId": "49109",
|
||||
"docId": "13644",
|
||||
"image_path": "data/docvqa_images/q49109_d13644.png",
|
||||
"ucsf_document_id": "mzdv0228",
|
||||
"ucsf_document_page_no": "1",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "46123",
|
||||
"questionId": "46123",
|
||||
"docId": "13503",
|
||||
"image_path": "data/docvqa_images/q46123_d13503.png",
|
||||
"ucsf_document_id": "xmww0217",
|
||||
"ucsf_document_page_no": "17",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "48158",
|
||||
"questionId": "48158",
|
||||
"docId": "13976",
|
||||
"image_path": "data/docvqa_images/q48158_d13976.png",
|
||||
"ucsf_document_id": "zqhm0227",
|
||||
"ucsf_document_page_no": "1",
|
||||
"topic": "free_text",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "1955",
|
||||
"questionId": "1955",
|
||||
"docId": "892",
|
||||
"image_path": "data/docvqa_images/q1955_d892.png",
|
||||
"ucsf_document_id": "jsbn0226",
|
||||
"ucsf_document_page_no": "2",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "8127",
|
||||
"questionId": "8127",
|
||||
"docId": "2754",
|
||||
"image_path": "data/docvqa_images/q8127_d2754.png",
|
||||
"ucsf_document_id": "xtvn0020",
|
||||
"ucsf_document_page_no": "2",
|
||||
"topic": "form",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "57431",
|
||||
"questionId": "57431",
|
||||
"docId": "4779",
|
||||
"image_path": "data/docvqa_images/q57431_d4779.png",
|
||||
"ucsf_document_id": "rnbx0223",
|
||||
"ucsf_document_page_no": "208",
|
||||
"topic": "Image/Photo",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "64306",
|
||||
"questionId": "64306",
|
||||
"docId": "10149",
|
||||
"image_path": "data/docvqa_images/q64306_d10149.png",
|
||||
"ucsf_document_id": "lpjm0223",
|
||||
"ucsf_document_page_no": "23",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "64887",
|
||||
"questionId": "64887",
|
||||
"docId": "9754",
|
||||
"image_path": "data/docvqa_images/q64887_d9754.png",
|
||||
"ucsf_document_id": "szpg0227",
|
||||
"ucsf_document_page_no": "9",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "58680",
|
||||
"questionId": "58680",
|
||||
"docId": "5545",
|
||||
"image_path": "data/docvqa_images/q58680_d5545.png",
|
||||
"ucsf_document_id": "hhwh0078",
|
||||
"ucsf_document_page_no": "1",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "5287",
|
||||
"questionId": "5287",
|
||||
"docId": "1785",
|
||||
"image_path": "data/docvqa_images/q5287_d1785.png",
|
||||
"ucsf_document_id": "mtnh0227",
|
||||
"ucsf_document_page_no": "10",
|
||||
"topic": "form",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "55471",
|
||||
"questionId": "55471",
|
||||
"docId": "4340",
|
||||
"image_path": "data/docvqa_images/q55471_d4340.png",
|
||||
"ucsf_document_id": "fsgj0223",
|
||||
"ucsf_document_page_no": "96",
|
||||
"topic": "free_text",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "53095",
|
||||
"questionId": "53095",
|
||||
"docId": "296",
|
||||
"image_path": "data/docvqa_images/q53095_d296.png",
|
||||
"ucsf_document_id": "qhxj0037",
|
||||
"ucsf_document_page_no": "3",
|
||||
"topic": "layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "53726",
|
||||
"questionId": "53726",
|
||||
"docId": "2008",
|
||||
"image_path": "data/docvqa_images/q53726_d2008.png",
|
||||
"ucsf_document_id": "hhnf0094",
|
||||
"ucsf_document_page_no": "5",
|
||||
"topic": "layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "57321",
|
||||
"questionId": "57321",
|
||||
"docId": "4722",
|
||||
"image_path": "data/docvqa_images/q57321_d4722.png",
|
||||
"ucsf_document_id": "xybx0223",
|
||||
"ucsf_document_page_no": "32",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "26659",
|
||||
"questionId": "26659",
|
||||
"docId": "7470",
|
||||
"image_path": "data/docvqa_images/q26659_d7470.png",
|
||||
"ucsf_document_id": "lhmg0227",
|
||||
"ucsf_document_page_no": "1",
|
||||
"topic": "layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "38920",
|
||||
"questionId": "38920",
|
||||
"docId": "11157",
|
||||
"image_path": "data/docvqa_images/q38920_d11157.png",
|
||||
"ucsf_document_id": "klnf0227",
|
||||
"ucsf_document_page_no": "1",
|
||||
"topic": "table/list|layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "50837",
|
||||
"questionId": "50837",
|
||||
"docId": "14742",
|
||||
"image_path": "data/docvqa_images/q50837_d14742.png",
|
||||
"ucsf_document_id": "ysmc0228",
|
||||
"ucsf_document_page_no": "4",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "59615",
|
||||
"questionId": "59615",
|
||||
"docId": "6569",
|
||||
"image_path": "data/docvqa_images/q59615_d6569.png",
|
||||
"ucsf_document_id": "hnnp0227",
|
||||
"ucsf_document_page_no": "45",
|
||||
"topic": "handwritten|table/list|layout",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
},
|
||||
{
|
||||
"id": "58687",
|
||||
"questionId": "58687",
|
||||
"docId": "5545",
|
||||
"image_path": "data/docvqa_images/q58687_d5545.png",
|
||||
"ucsf_document_id": "hhwh0078",
|
||||
"ucsf_document_page_no": "1",
|
||||
"topic": "table/list",
|
||||
"source_dataset": "lmms-lab/DocVQA",
|
||||
"source_config": "DocVQA",
|
||||
"source_split": "validation",
|
||||
"sample_seed": "full_validation_5349"
|
||||
}
|
||||
]
|
||||
34
data/livemathematicianbench_id_split/split_manifest.json
Normal file
34
data/livemathematicianbench_id_split/split_manifest.json
Normal file
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"benchmark": "LiveMathematicianBench",
|
||||
"manifest_type": "id_split",
|
||||
"source_repo": "LiveMathematicianBench/LiveMathematicianBench",
|
||||
"source_repo_type": "dataset",
|
||||
"source_url": "https://huggingface.co/datasets/LiveMathematicianBench/LiveMathematicianBench",
|
||||
"source_revision": "b72450f6ce96c26158d64d945a5d31ef7727be41",
|
||||
"source_files": [
|
||||
"data/202511/qa_202511_final.json",
|
||||
"data/202512/qa_202512_final.json",
|
||||
"data/202601/qa_202601_final.json",
|
||||
"data/202602/qa_202602_final.json"
|
||||
],
|
||||
"split_mode": "ratio",
|
||||
"split_ratio": "2:1:7",
|
||||
"split_seed": 42,
|
||||
"counts": {
|
||||
"train": 35,
|
||||
"val": 18,
|
||||
"test": 124
|
||||
},
|
||||
"item_fields": [
|
||||
"id",
|
||||
"month",
|
||||
"no",
|
||||
"paper_link",
|
||||
"source_file"
|
||||
],
|
||||
"id_format": "<month>:<no>",
|
||||
"notes": [
|
||||
"This is an ID split manifest, not the full LiveMathematicianBench payload.",
|
||||
"Materialize full split items from the official LiveMathematicianBench raw qa_*_final.json files before evaluation."
|
||||
]
|
||||
}
|
||||
870
data/livemathematicianbench_id_split/test/items.json
Normal file
870
data/livemathematicianbench_id_split/test/items.json
Normal file
@@ -0,0 +1,870 @@
|
||||
[
|
||||
{
|
||||
"id": "202602:12",
|
||||
"month": "202602",
|
||||
"no": 12,
|
||||
"paper_link": "http://arxiv.org/abs/2602.07171v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:3",
|
||||
"month": "202601",
|
||||
"no": 3,
|
||||
"paper_link": "http://arxiv.org/abs/2601.01447v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:4",
|
||||
"month": "202511",
|
||||
"no": 4,
|
||||
"paper_link": "http://arxiv.org/abs/2511.23123v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:20",
|
||||
"month": "202601",
|
||||
"no": 20,
|
||||
"paper_link": "http://arxiv.org/abs/2601.13212v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:42",
|
||||
"month": "202601",
|
||||
"no": 42,
|
||||
"paper_link": "http://arxiv.org/abs/2601.09348v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:38",
|
||||
"month": "202512",
|
||||
"no": 38,
|
||||
"paper_link": "http://arxiv.org/abs/2512.19831v2",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:4",
|
||||
"month": "202512",
|
||||
"no": 4,
|
||||
"paper_link": "http://arxiv.org/abs/2512.03141v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:4",
|
||||
"month": "202602",
|
||||
"no": 4,
|
||||
"paper_link": "http://arxiv.org/abs/2602.14368v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:15",
|
||||
"month": "202511",
|
||||
"no": 15,
|
||||
"paper_link": "http://arxiv.org/abs/2511.17325v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:32",
|
||||
"month": "202602",
|
||||
"no": 32,
|
||||
"paper_link": "http://arxiv.org/abs/2602.14817v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:51",
|
||||
"month": "202512",
|
||||
"no": 51,
|
||||
"paper_link": "http://arxiv.org/abs/2512.14581v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:26",
|
||||
"month": "202512",
|
||||
"no": 26,
|
||||
"paper_link": "http://arxiv.org/abs/2512.19586v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:13",
|
||||
"month": "202601",
|
||||
"no": 13,
|
||||
"paper_link": "http://arxiv.org/abs/2601.10017v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:1",
|
||||
"month": "202602",
|
||||
"no": 1,
|
||||
"paper_link": "http://arxiv.org/abs/2602.23137v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:18",
|
||||
"month": "202511",
|
||||
"no": 18,
|
||||
"paper_link": "http://arxiv.org/abs/2511.10795v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:5",
|
||||
"month": "202512",
|
||||
"no": 5,
|
||||
"paper_link": "http://arxiv.org/abs/2512.00348v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:19",
|
||||
"month": "202511",
|
||||
"no": 19,
|
||||
"paper_link": "http://arxiv.org/abs/2511.06951v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:40",
|
||||
"month": "202602",
|
||||
"no": 40,
|
||||
"paper_link": "http://arxiv.org/abs/2602.20462v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:29",
|
||||
"month": "202602",
|
||||
"no": 29,
|
||||
"paper_link": "http://arxiv.org/abs/2602.10676v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:35",
|
||||
"month": "202512",
|
||||
"no": 35,
|
||||
"paper_link": "http://arxiv.org/abs/2512.08840v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:48",
|
||||
"month": "202512",
|
||||
"no": 48,
|
||||
"paper_link": "http://arxiv.org/abs/2512.03482v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:52",
|
||||
"month": "202512",
|
||||
"no": 52,
|
||||
"paper_link": "http://arxiv.org/abs/2512.11246v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:44",
|
||||
"month": "202512",
|
||||
"no": 44,
|
||||
"paper_link": "http://arxiv.org/abs/2512.10385v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:28",
|
||||
"month": "202511",
|
||||
"no": 28,
|
||||
"paper_link": "http://arxiv.org/abs/2511.03812v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:43",
|
||||
"month": "202601",
|
||||
"no": 43,
|
||||
"paper_link": "http://arxiv.org/abs/2601.22555v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:9",
|
||||
"month": "202602",
|
||||
"no": 9,
|
||||
"paper_link": "http://arxiv.org/abs/2602.19882v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:23",
|
||||
"month": "202512",
|
||||
"no": 23,
|
||||
"paper_link": "http://arxiv.org/abs/2512.09180v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:21",
|
||||
"month": "202602",
|
||||
"no": 21,
|
||||
"paper_link": "http://arxiv.org/abs/2602.10509v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:5",
|
||||
"month": "202511",
|
||||
"no": 5,
|
||||
"paper_link": "http://arxiv.org/abs/2511.20164v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:35",
|
||||
"month": "202601",
|
||||
"no": 35,
|
||||
"paper_link": "http://arxiv.org/abs/2601.15606v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:50",
|
||||
"month": "202602",
|
||||
"no": 50,
|
||||
"paper_link": "http://arxiv.org/abs/2602.05652v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:13",
|
||||
"month": "202512",
|
||||
"no": 13,
|
||||
"paper_link": "http://arxiv.org/abs/2512.22861v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:49",
|
||||
"month": "202602",
|
||||
"no": 49,
|
||||
"paper_link": "http://arxiv.org/abs/2602.07167v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:18",
|
||||
"month": "202602",
|
||||
"no": 18,
|
||||
"paper_link": "http://arxiv.org/abs/2602.20124v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:15",
|
||||
"month": "202601",
|
||||
"no": 15,
|
||||
"paper_link": "http://arxiv.org/abs/2601.05327v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:21",
|
||||
"month": "202601",
|
||||
"no": 21,
|
||||
"paper_link": "http://arxiv.org/abs/2601.04994v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:32",
|
||||
"month": "202601",
|
||||
"no": 32,
|
||||
"paper_link": "http://arxiv.org/abs/2601.09183v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:34",
|
||||
"month": "202602",
|
||||
"no": 34,
|
||||
"paper_link": "http://arxiv.org/abs/2602.21118v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:20",
|
||||
"month": "202602",
|
||||
"no": 20,
|
||||
"paper_link": "http://arxiv.org/abs/2602.16506v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:5",
|
||||
"month": "202602",
|
||||
"no": 5,
|
||||
"paper_link": "http://arxiv.org/abs/2602.09806v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:40",
|
||||
"month": "202512",
|
||||
"no": 40,
|
||||
"paper_link": "http://arxiv.org/abs/2512.16535v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:22",
|
||||
"month": "202511",
|
||||
"no": 22,
|
||||
"paper_link": "http://arxiv.org/abs/2511.07607v2",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:36",
|
||||
"month": "202601",
|
||||
"no": 36,
|
||||
"paper_link": "http://arxiv.org/abs/2601.12457v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:49",
|
||||
"month": "202512",
|
||||
"no": 49,
|
||||
"paper_link": "http://arxiv.org/abs/2512.21565v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:10",
|
||||
"month": "202511",
|
||||
"no": 10,
|
||||
"paper_link": "http://arxiv.org/abs/2511.06484v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:2",
|
||||
"month": "202601",
|
||||
"no": 2,
|
||||
"paper_link": "http://arxiv.org/abs/2601.07068v4",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:19",
|
||||
"month": "202602",
|
||||
"no": 19,
|
||||
"paper_link": "http://arxiv.org/abs/2602.18179v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:9",
|
||||
"month": "202601",
|
||||
"no": 9,
|
||||
"paper_link": "http://arxiv.org/abs/2601.17765v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:6",
|
||||
"month": "202512",
|
||||
"no": 6,
|
||||
"paper_link": "http://arxiv.org/abs/2512.23079v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:5",
|
||||
"month": "202601",
|
||||
"no": 5,
|
||||
"paper_link": "http://arxiv.org/abs/2601.20344v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:14",
|
||||
"month": "202602",
|
||||
"no": 14,
|
||||
"paper_link": "http://arxiv.org/abs/2602.09177v2",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:17",
|
||||
"month": "202512",
|
||||
"no": 17,
|
||||
"paper_link": "http://arxiv.org/abs/2512.11657v2",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:19",
|
||||
"month": "202512",
|
||||
"no": 19,
|
||||
"paper_link": "http://arxiv.org/abs/2512.16655v2",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:33",
|
||||
"month": "202602",
|
||||
"no": 33,
|
||||
"paper_link": "http://arxiv.org/abs/2602.13734v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:18",
|
||||
"month": "202512",
|
||||
"no": 18,
|
||||
"paper_link": "http://arxiv.org/abs/2512.22960v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:26",
|
||||
"month": "202601",
|
||||
"no": 26,
|
||||
"paper_link": "http://arxiv.org/abs/2601.06814v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:1",
|
||||
"month": "202601",
|
||||
"no": 1,
|
||||
"paper_link": "http://arxiv.org/abs/2601.18276v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:30",
|
||||
"month": "202512",
|
||||
"no": 30,
|
||||
"paper_link": "http://arxiv.org/abs/2512.07260v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:44",
|
||||
"month": "202602",
|
||||
"no": 44,
|
||||
"paper_link": "http://arxiv.org/abs/2602.01138v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:20",
|
||||
"month": "202512",
|
||||
"no": 20,
|
||||
"paper_link": "http://arxiv.org/abs/2512.14575v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:13",
|
||||
"month": "202511",
|
||||
"no": 13,
|
||||
"paper_link": "http://arxiv.org/abs/2511.16910v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:30",
|
||||
"month": "202601",
|
||||
"no": 30,
|
||||
"paper_link": "http://arxiv.org/abs/2601.12140v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:40",
|
||||
"month": "202601",
|
||||
"no": 40,
|
||||
"paper_link": "http://arxiv.org/abs/2601.05146v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:29",
|
||||
"month": "202601",
|
||||
"no": 29,
|
||||
"paper_link": "http://arxiv.org/abs/2601.12846v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:11",
|
||||
"month": "202511",
|
||||
"no": 11,
|
||||
"paper_link": "http://arxiv.org/abs/2511.17548v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:9",
|
||||
"month": "202512",
|
||||
"no": 9,
|
||||
"paper_link": "http://arxiv.org/abs/2512.08817v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:18",
|
||||
"month": "202601",
|
||||
"no": 18,
|
||||
"paper_link": "http://arxiv.org/abs/2601.01797v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:1",
|
||||
"month": "202512",
|
||||
"no": 1,
|
||||
"paper_link": "http://arxiv.org/abs/2512.20055v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:4",
|
||||
"month": "202601",
|
||||
"no": 4,
|
||||
"paper_link": "http://arxiv.org/abs/2601.21223v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:6",
|
||||
"month": "202511",
|
||||
"no": 6,
|
||||
"paper_link": "http://arxiv.org/abs/2511.14959v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:38",
|
||||
"month": "202602",
|
||||
"no": 38,
|
||||
"paper_link": "http://arxiv.org/abs/2602.08398v2",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:10",
|
||||
"month": "202601",
|
||||
"no": 10,
|
||||
"paper_link": "http://arxiv.org/abs/2601.15524v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:11",
|
||||
"month": "202602",
|
||||
"no": 11,
|
||||
"paper_link": "http://arxiv.org/abs/2602.11045v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:45",
|
||||
"month": "202512",
|
||||
"no": 45,
|
||||
"paper_link": "http://arxiv.org/abs/2512.08395v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:12",
|
||||
"month": "202601",
|
||||
"no": 12,
|
||||
"paper_link": "http://arxiv.org/abs/2601.11877v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:47",
|
||||
"month": "202512",
|
||||
"no": 47,
|
||||
"paper_link": "http://arxiv.org/abs/2512.09683v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:21",
|
||||
"month": "202511",
|
||||
"no": 21,
|
||||
"paper_link": "http://arxiv.org/abs/2511.21288v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:16",
|
||||
"month": "202601",
|
||||
"no": 16,
|
||||
"paper_link": "http://arxiv.org/abs/2601.05008v2",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:3",
|
||||
"month": "202512",
|
||||
"no": 3,
|
||||
"paper_link": "http://arxiv.org/abs/2512.13450v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:37",
|
||||
"month": "202601",
|
||||
"no": 37,
|
||||
"paper_link": "http://arxiv.org/abs/2601.09443v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:12",
|
||||
"month": "202511",
|
||||
"no": 12,
|
||||
"paper_link": "http://arxiv.org/abs/2511.04978v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:39",
|
||||
"month": "202512",
|
||||
"no": 39,
|
||||
"paper_link": "http://arxiv.org/abs/2512.19003v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:8",
|
||||
"month": "202601",
|
||||
"no": 8,
|
||||
"paper_link": "http://arxiv.org/abs/2601.19754v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:11",
|
||||
"month": "202601",
|
||||
"no": 11,
|
||||
"paper_link": "http://arxiv.org/abs/2601.13552v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:25",
|
||||
"month": "202511",
|
||||
"no": 25,
|
||||
"paper_link": "http://arxiv.org/abs/2511.10548v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:17",
|
||||
"month": "202601",
|
||||
"no": 17,
|
||||
"paper_link": "http://arxiv.org/abs/2601.02655v2",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:36",
|
||||
"month": "202602",
|
||||
"no": 36,
|
||||
"paper_link": "http://arxiv.org/abs/2602.13001v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:43",
|
||||
"month": "202602",
|
||||
"no": 43,
|
||||
"paper_link": "http://arxiv.org/abs/2602.06897v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:6",
|
||||
"month": "202601",
|
||||
"no": 6,
|
||||
"paper_link": "http://arxiv.org/abs/2601.04747v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:35",
|
||||
"month": "202602",
|
||||
"no": 35,
|
||||
"paper_link": "http://arxiv.org/abs/2602.20938v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:11",
|
||||
"month": "202512",
|
||||
"no": 11,
|
||||
"paper_link": "http://arxiv.org/abs/2512.03294v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:23",
|
||||
"month": "202602",
|
||||
"no": 23,
|
||||
"paper_link": "http://arxiv.org/abs/2602.09201v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:7",
|
||||
"month": "202601",
|
||||
"no": 7,
|
||||
"paper_link": "http://arxiv.org/abs/2601.02859v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:39",
|
||||
"month": "202602",
|
||||
"no": 39,
|
||||
"paper_link": "http://arxiv.org/abs/2602.21659v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:12",
|
||||
"month": "202512",
|
||||
"no": 12,
|
||||
"paper_link": "http://arxiv.org/abs/2512.00690v3",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:2",
|
||||
"month": "202511",
|
||||
"no": 2,
|
||||
"paper_link": "http://arxiv.org/abs/2511.19681v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:43",
|
||||
"month": "202512",
|
||||
"no": 43,
|
||||
"paper_link": "http://arxiv.org/abs/2512.10820v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:24",
|
||||
"month": "202602",
|
||||
"no": 24,
|
||||
"paper_link": "http://arxiv.org/abs/2602.08680v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:34",
|
||||
"month": "202601",
|
||||
"no": 34,
|
||||
"paper_link": "http://arxiv.org/abs/2601.07318v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:28",
|
||||
"month": "202512",
|
||||
"no": 28,
|
||||
"paper_link": "http://arxiv.org/abs/2512.11294v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:27",
|
||||
"month": "202601",
|
||||
"no": 27,
|
||||
"paper_link": "http://arxiv.org/abs/2601.05692v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:42",
|
||||
"month": "202602",
|
||||
"no": 42,
|
||||
"paper_link": "http://arxiv.org/abs/2602.09749v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:22",
|
||||
"month": "202512",
|
||||
"no": 22,
|
||||
"paper_link": "http://arxiv.org/abs/2512.11658v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:17",
|
||||
"month": "202602",
|
||||
"no": 17,
|
||||
"paper_link": "http://arxiv.org/abs/2602.22504v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:48",
|
||||
"month": "202602",
|
||||
"no": 48,
|
||||
"paper_link": "http://arxiv.org/abs/2602.08760v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:28",
|
||||
"month": "202602",
|
||||
"no": 28,
|
||||
"paper_link": "http://arxiv.org/abs/2602.11595v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:3",
|
||||
"month": "202602",
|
||||
"no": 3,
|
||||
"paper_link": "http://arxiv.org/abs/2602.17369v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:31",
|
||||
"month": "202512",
|
||||
"no": 31,
|
||||
"paper_link": "http://arxiv.org/abs/2512.23668v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:27",
|
||||
"month": "202512",
|
||||
"no": 27,
|
||||
"paper_link": "http://arxiv.org/abs/2512.16505v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:24",
|
||||
"month": "202511",
|
||||
"no": 24,
|
||||
"paper_link": "http://arxiv.org/abs/2511.12549v2",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:8",
|
||||
"month": "202511",
|
||||
"no": 8,
|
||||
"paper_link": "http://arxiv.org/abs/2511.12657v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:9",
|
||||
"month": "202511",
|
||||
"no": 9,
|
||||
"paper_link": "http://arxiv.org/abs/2511.09015v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:28",
|
||||
"month": "202601",
|
||||
"no": 28,
|
||||
"paper_link": "http://arxiv.org/abs/2601.14825v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:25",
|
||||
"month": "202602",
|
||||
"no": 25,
|
||||
"paper_link": "http://arxiv.org/abs/2602.16048v3",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:23",
|
||||
"month": "202511",
|
||||
"no": 23,
|
||||
"paper_link": "http://arxiv.org/abs/2511.06595v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:13",
|
||||
"month": "202602",
|
||||
"no": 13,
|
||||
"paper_link": "http://arxiv.org/abs/2602.12261v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:27",
|
||||
"month": "202511",
|
||||
"no": 27,
|
||||
"paper_link": "http://arxiv.org/abs/2511.04407v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:7",
|
||||
"month": "202512",
|
||||
"no": 7,
|
||||
"paper_link": "http://arxiv.org/abs/2512.09490v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:29",
|
||||
"month": "202512",
|
||||
"no": 29,
|
||||
"paper_link": "http://arxiv.org/abs/2512.08562v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:34",
|
||||
"month": "202512",
|
||||
"no": 34,
|
||||
"paper_link": "http://arxiv.org/abs/2512.09598v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:42",
|
||||
"month": "202512",
|
||||
"no": 42,
|
||||
"paper_link": "http://arxiv.org/abs/2512.10845v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:7",
|
||||
"month": "202511",
|
||||
"no": 7,
|
||||
"paper_link": "http://arxiv.org/abs/2511.13976v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:29",
|
||||
"month": "202511",
|
||||
"no": 29,
|
||||
"paper_link": "http://arxiv.org/abs/2511.03722v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:37",
|
||||
"month": "202602",
|
||||
"no": 37,
|
||||
"paper_link": "http://arxiv.org/abs/2602.08644v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
}
|
||||
]
|
||||
247
data/livemathematicianbench_id_split/train/items.json
Normal file
247
data/livemathematicianbench_id_split/train/items.json
Normal file
@@ -0,0 +1,247 @@
|
||||
[
|
||||
{
|
||||
"id": "202602:22",
|
||||
"month": "202602",
|
||||
"no": 22,
|
||||
"paper_link": "http://arxiv.org/abs/2602.10700v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:8",
|
||||
"month": "202512",
|
||||
"no": 8,
|
||||
"paper_link": "http://arxiv.org/abs/2512.08863v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:16",
|
||||
"month": "202511",
|
||||
"no": 16,
|
||||
"paper_link": "http://arxiv.org/abs/2511.15668v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:44",
|
||||
"month": "202601",
|
||||
"no": 44,
|
||||
"paper_link": "http://arxiv.org/abs/2601.21267v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:14",
|
||||
"month": "202511",
|
||||
"no": 14,
|
||||
"paper_link": "http://arxiv.org/abs/2511.13447v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:30",
|
||||
"month": "202602",
|
||||
"no": 30,
|
||||
"paper_link": "http://arxiv.org/abs/2602.16692v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:2",
|
||||
"month": "202602",
|
||||
"no": 2,
|
||||
"paper_link": "http://arxiv.org/abs/2602.22933v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:41",
|
||||
"month": "202601",
|
||||
"no": 41,
|
||||
"paper_link": "http://arxiv.org/abs/2601.01164v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:23",
|
||||
"month": "202601",
|
||||
"no": 23,
|
||||
"paper_link": "http://arxiv.org/abs/2601.02528v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:20",
|
||||
"month": "202511",
|
||||
"no": 20,
|
||||
"paper_link": "http://arxiv.org/abs/2511.02963v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:22",
|
||||
"month": "202601",
|
||||
"no": 22,
|
||||
"paper_link": "http://arxiv.org/abs/2601.03984v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:14",
|
||||
"month": "202512",
|
||||
"no": 14,
|
||||
"paper_link": "http://arxiv.org/abs/2512.22459v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:26",
|
||||
"month": "202511",
|
||||
"no": 26,
|
||||
"paper_link": "http://arxiv.org/abs/2511.07817v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:3",
|
||||
"month": "202511",
|
||||
"no": 3,
|
||||
"paper_link": "http://arxiv.org/abs/2511.11409v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:33",
|
||||
"month": "202601",
|
||||
"no": 33,
|
||||
"paper_link": "http://arxiv.org/abs/2601.07747v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:7",
|
||||
"month": "202602",
|
||||
"no": 7,
|
||||
"paper_link": "http://arxiv.org/abs/2602.22912v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:27",
|
||||
"month": "202602",
|
||||
"no": 27,
|
||||
"paper_link": "http://arxiv.org/abs/2602.13968v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:31",
|
||||
"month": "202602",
|
||||
"no": 31,
|
||||
"paper_link": "http://arxiv.org/abs/2602.15528v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:41",
|
||||
"month": "202602",
|
||||
"no": 41,
|
||||
"paper_link": "http://arxiv.org/abs/2602.10707v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:25",
|
||||
"month": "202512",
|
||||
"no": 25,
|
||||
"paper_link": "http://arxiv.org/abs/2512.04531v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:10",
|
||||
"month": "202602",
|
||||
"no": 10,
|
||||
"paper_link": "http://arxiv.org/abs/2602.17863v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:16",
|
||||
"month": "202602",
|
||||
"no": 16,
|
||||
"paper_link": "http://arxiv.org/abs/2602.02723v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:16",
|
||||
"month": "202512",
|
||||
"no": 16,
|
||||
"paper_link": "http://arxiv.org/abs/2512.11601v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:2",
|
||||
"month": "202512",
|
||||
"no": 2,
|
||||
"paper_link": "http://arxiv.org/abs/2512.16120v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:24",
|
||||
"month": "202512",
|
||||
"no": 24,
|
||||
"paper_link": "http://arxiv.org/abs/2512.08391v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:32",
|
||||
"month": "202512",
|
||||
"no": 32,
|
||||
"paper_link": "http://arxiv.org/abs/2512.23224v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:47",
|
||||
"month": "202602",
|
||||
"no": 47,
|
||||
"paper_link": "http://arxiv.org/abs/2602.10391v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:46",
|
||||
"month": "202602",
|
||||
"no": 46,
|
||||
"paper_link": "http://arxiv.org/abs/2602.13727v2",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:21",
|
||||
"month": "202512",
|
||||
"no": 21,
|
||||
"paper_link": "http://arxiv.org/abs/2512.12835v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:33",
|
||||
"month": "202512",
|
||||
"no": 33,
|
||||
"paper_link": "http://arxiv.org/abs/2512.19500v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:45",
|
||||
"month": "202602",
|
||||
"no": 45,
|
||||
"paper_link": "http://arxiv.org/abs/2602.23912v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:26",
|
||||
"month": "202602",
|
||||
"no": 26,
|
||||
"paper_link": "http://arxiv.org/abs/2602.14658v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:41",
|
||||
"month": "202512",
|
||||
"no": 41,
|
||||
"paper_link": "http://arxiv.org/abs/2512.15177v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:38",
|
||||
"month": "202601",
|
||||
"no": 38,
|
||||
"paper_link": "http://arxiv.org/abs/2601.07817v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:14",
|
||||
"month": "202601",
|
||||
"no": 14,
|
||||
"paper_link": "http://arxiv.org/abs/2601.08704v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
}
|
||||
]
|
||||
128
data/livemathematicianbench_id_split/val/items.json
Normal file
128
data/livemathematicianbench_id_split/val/items.json
Normal file
@@ -0,0 +1,128 @@
|
||||
[
|
||||
{
|
||||
"id": "202602:8",
|
||||
"month": "202602",
|
||||
"no": 8,
|
||||
"paper_link": "http://arxiv.org/abs/2602.19529v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:50",
|
||||
"month": "202512",
|
||||
"no": 50,
|
||||
"paper_link": "http://arxiv.org/abs/2512.15277v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:36",
|
||||
"month": "202512",
|
||||
"no": 36,
|
||||
"paper_link": "http://arxiv.org/abs/2512.06696v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:1",
|
||||
"month": "202511",
|
||||
"no": 1,
|
||||
"paper_link": "http://arxiv.org/abs/2511.04651v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:31",
|
||||
"month": "202601",
|
||||
"no": 31,
|
||||
"paper_link": "http://arxiv.org/abs/2601.10298v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202511:17",
|
||||
"month": "202511",
|
||||
"no": 17,
|
||||
"paper_link": "http://arxiv.org/abs/2511.13215v1",
|
||||
"source_file": "data/202511/qa_202511_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:37",
|
||||
"month": "202512",
|
||||
"no": 37,
|
||||
"paper_link": "http://arxiv.org/abs/2512.20498v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:39",
|
||||
"month": "202601",
|
||||
"no": 39,
|
||||
"paper_link": "http://arxiv.org/abs/2601.06601v2",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:25",
|
||||
"month": "202601",
|
||||
"no": 25,
|
||||
"paper_link": "http://arxiv.org/abs/2601.10996v3",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:24",
|
||||
"month": "202601",
|
||||
"no": 24,
|
||||
"paper_link": "http://arxiv.org/abs/2601.12250v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:45",
|
||||
"month": "202601",
|
||||
"no": 45,
|
||||
"paper_link": "http://arxiv.org/abs/2601.12113v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:19",
|
||||
"month": "202601",
|
||||
"no": 19,
|
||||
"paper_link": "http://arxiv.org/abs/2601.00779v1",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:10",
|
||||
"month": "202512",
|
||||
"no": 10,
|
||||
"paper_link": "http://arxiv.org/abs/2512.07073v2",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202601:46",
|
||||
"month": "202601",
|
||||
"no": 46,
|
||||
"paper_link": "http://arxiv.org/abs/2601.07793v2",
|
||||
"source_file": "data/202601/qa_202601_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:15",
|
||||
"month": "202512",
|
||||
"no": 15,
|
||||
"paper_link": "http://arxiv.org/abs/2512.16165v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:15",
|
||||
"month": "202602",
|
||||
"no": 15,
|
||||
"paper_link": "http://arxiv.org/abs/2602.05303v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202602:6",
|
||||
"month": "202602",
|
||||
"no": 6,
|
||||
"paper_link": "http://arxiv.org/abs/2602.01571v1",
|
||||
"source_file": "data/202602/qa_202602_final.json"
|
||||
},
|
||||
{
|
||||
"id": "202512:46",
|
||||
"month": "202512",
|
||||
"no": 46,
|
||||
"paper_link": "http://arxiv.org/abs/2512.05945v1",
|
||||
"source_file": "data/202512/qa_202512_final.json"
|
||||
}
|
||||
]
|
||||
27
data/officeqa_id_split/split_manifest.json
Normal file
27
data/officeqa_id_split/split_manifest.json
Normal file
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"benchmark": "OfficeQA",
|
||||
"manifest_type": "id_split",
|
||||
"source_repo": "databricks/officeqa",
|
||||
"source_repo_type": "dataset",
|
||||
"source_url": "https://huggingface.co/datasets/databricks/officeqa",
|
||||
"source_revision": "8ecbf18d3833daf4750a903d14963e4c4c1d4cd8",
|
||||
"source_file": "officeqa_full.csv",
|
||||
"source_split_name": "officeqa_split",
|
||||
"counts": {
|
||||
"train": 50,
|
||||
"val": 24,
|
||||
"test": 172
|
||||
},
|
||||
"item_fields": [
|
||||
"id",
|
||||
"uid",
|
||||
"category",
|
||||
"source_files",
|
||||
"source_docs",
|
||||
"source_split"
|
||||
],
|
||||
"notes": [
|
||||
"This is a split manifest, not the full OfficeQA payload.",
|
||||
"The official OfficeQA CSV is gated on Hugging Face; materialization requires authorized access."
|
||||
]
|
||||
}
|
||||
1378
data/officeqa_id_split/test/items.json
Normal file
1378
data/officeqa_id_split/test/items.json
Normal file
File diff suppressed because it is too large
Load Diff
402
data/officeqa_id_split/train/items.json
Normal file
402
data/officeqa_id_split/train/items.json
Normal file
@@ -0,0 +1,402 @@
|
||||
[
|
||||
{
|
||||
"id": "UID0002",
|
||||
"uid": "UID0002",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1944_01.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/january-1944-6565?page=18",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0007",
|
||||
"uid": "UID0007",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1950_02.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/february-1950-6637?page=15",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0014",
|
||||
"uid": "UID0014",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1942_07.txt\r\ntreasury_bulletin_2001_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/july-1942-6547?page=76\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2001-7108?page=17&deep=true",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0017",
|
||||
"uid": "UID0017",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1982_08.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/august-1982-7028?page=13",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0018",
|
||||
"uid": "UID0018",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1985_03.txt\r\ntreasury_bulletin_1986_03.txt\r\ntreasury_bulletin_1987_03.txt\r\ntreasury_bulletin_1988_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1985-7040?page=22\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1986-7045?page=26\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1987-7049?page=24\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1988-7052?page=36",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0019",
|
||||
"uid": "UID0019",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_2016_09.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/september-2016-533966?page=54\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/september-2016-533966?page=58",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0026",
|
||||
"uid": "UID0026",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1963_01.txt\r\ntreasury_bulletin_1962_01.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/january-1963-6793?page=88\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/january-1962-6781?page=82&deep=true",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0028",
|
||||
"uid": "UID0028",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1970_06.txt\r\ntreasury_bulletin_1964_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/june-1970-6882?page=89&deep=true\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/december-1964-6816?page=25&deep=true",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0030",
|
||||
"uid": "UID0030",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1990_09.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/september-1990-7062?page=19&deep=true",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0031",
|
||||
"uid": "UID0031",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1992_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1992-7068?page=158&deep=true",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0033",
|
||||
"uid": "UID0033",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1977_04.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1977-6964?page=9",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0034",
|
||||
"uid": "UID0034",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1992_06.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/june-1992-7069?page=32",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0044",
|
||||
"uid": "UID0044",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1939_04.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1939-6506?page=61",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0046",
|
||||
"uid": "UID0046",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1988_09.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/september-1988-7054?page=37",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0049",
|
||||
"uid": "UID0049",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1942_02.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/february-1942-6542?page=19&deep=true",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0056",
|
||||
"uid": "UID0056",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1991_09.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/september-1991-7066?page=30&deep=true",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0063",
|
||||
"uid": "UID0063",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1990_06.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/june-1990-7061?page=127",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0065",
|
||||
"uid": "UID0065",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1998_06.txt\r\ntreasury_bulletin_1995_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/june-1998-7094?page=7\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/december-1995-7084?page=16",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0073",
|
||||
"uid": "UID0073",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1982_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1982-7023?page=24",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0079",
|
||||
"uid": "UID0079",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_2011_12.txt\r\ntreasury_bulletin_2013_12.txt\r\ntreasury_bulletin_2015_12.txt\r\ntreasury_bulletin_2017_12.txt\r\ntreasury_bulletin_2019_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2011-7149?page=25\r\n \r\n https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2013-7155?page=24\r\n \r\n https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2015-519209?page=23\r\n \r\n https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2017-575188?page=23\r\n \r\n https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2019-584842?page=22",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0083",
|
||||
"uid": "UID0083",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1981_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-1981-7020?page=24",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0085",
|
||||
"uid": "UID0085",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_2019_12.txt\r\ntreasury_bulletin_2018_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2019-584842?page=23\r\n\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2018-581283?page=22",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0087",
|
||||
"uid": "UID0087",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_2013_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2013-7155?page=17",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0092",
|
||||
"uid": "UID0092",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1987_12.txt\r\ntreasury_bulletin_1992_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-1987-7051?page=69\r\n\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/december-1992-7071?page=84",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0098",
|
||||
"uid": "UID0098",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_2020_12.txt\r\ntreasury_bulletin_2024_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2020-598551?page=21\r\n \r\n https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2024-679984?page=22",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0101",
|
||||
"uid": "UID0101",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_2011_12.txt\r\ntreasury_bulletin_2019_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2011-7149?page=25\r\n \r\n https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2019-584842?page=22",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0110",
|
||||
"uid": "UID0110",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_2020_03.txt\r\ntreasury_bulletin_2016_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-2020-587316?page=10\r\n \r\n https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-2016-527290?page=9",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0115",
|
||||
"uid": "UID0115",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1980_02.txt\r\ntreasury_bulletin_1981_02.txt\r\ntreasury_bulletin_1982_02.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/february-1980-6998?page=27\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/february-1981-7010?page=38\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/february-1982-7022?page=31",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0122",
|
||||
"uid": "UID0122",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_2001_03.txt\r\ntreasury_bulletin_2002_03.txt\r\ntreasury_bulletin_2003_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-2001-7105?page=112\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-2002-7110?page=115\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-2003-7109?page=113",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0123",
|
||||
"uid": "UID0123",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1941_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-1941-6540?page=91",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0133",
|
||||
"uid": "UID0133",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_2004_09.txt\r\ntreasury_bulletin_2013_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/september-2004-7119?page=63\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2013-7155?page=68",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0141",
|
||||
"uid": "UID0141",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1962_04.txt\r\ntreasury_bulletin_1963_04.txt\r\ntreasury_bulletin_1964_04.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1962-6784?page=75\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1963-6796?page=79\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1964-6808?page=82",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0144",
|
||||
"uid": "UID0144",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1980_11.txt\r\ntreasury_bulletin_1981_11.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/november-1980-7007?page=76\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/november-1981-7019?page=67",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0145",
|
||||
"uid": "UID0145",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1943_01.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/january-1943-6553?page=41",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0150",
|
||||
"uid": "UID0150",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1972_04.txt\r\ntreasury_bulletin_1973_04.txt\r\ntreasury_bulletin_1974_04.txt\r\ntreasury_bulletin_1975_04.txt\r\ntreasury_bulletin_1976_04.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1972-6905?page=89\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1973-6916?page=91\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1974-6927?page=88\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1975-6940?page=88\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1976-6952?page=104",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0151",
|
||||
"uid": "UID0151",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1953_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1953-6674?page=54",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0162",
|
||||
"uid": "UID0162",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_2011_06.txt\r\ntreasury_bulletin_2012_06.txt\r\ntreasury_bulletin_2013_06.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/june-2011-7129?page=105\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/june-2012-7151?page=105\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/june-2013-7153?page=104",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0163",
|
||||
"uid": "UID0163",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1981_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-1981-7020?page=28",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0165",
|
||||
"uid": "UID0165",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_2010_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-2010-7143?page=49",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0166",
|
||||
"uid": "UID0166",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1943_03.txt\r\ntreasury_bulletin_1944_03.txt\r\ntreasury_bulletin_1945_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1943-6555?page=68\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1944-6567?page=83\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1945-6577?page=71",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0169",
|
||||
"uid": "UID0169",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1982_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1982-7023?page=73",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0189",
|
||||
"uid": "UID0189",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1970_08.txt\r\ntreasury_bulletin_1970_09.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/august-1970-6884?page=70\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/september-1970-6885?page=70",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0195",
|
||||
"uid": "UID0195",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1956_08.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/august-1956-6715?page=59",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0202",
|
||||
"uid": "UID0202",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1939_07.txt\r\ntreasury_bulletin_1939_08.txt\r\ntreasury_bulletin_1939_09.txt\r\ntreasury_bulletin_1939_10.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/july-1939-6509?page=99\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/august-1939-6510?page=107\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/september-1939-6511?page=60\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/october-1939-6520?page=62",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0212",
|
||||
"uid": "UID0212",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1964_01.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/january-1964-6805?page=99",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0222",
|
||||
"uid": "UID0222",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_2001_06.txt\r\ntreasury_bulletin_2006_06.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/june-2001-7106?page=50\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/june-2006-7126?page=50",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0228",
|
||||
"uid": "UID0228",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1956_03.txt\r\ntreasury_bulletin_1956_04.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1956-6710?page=22\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1956-6711?page=22",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0229",
|
||||
"uid": "UID0229",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_2005_03.txt\r\ntreasury_bulletin_2006_03.txt\r\ntreasury_bulletin_2007_03.txt\r\ntreasury_bulletin_2008_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-2005-7121?page=109\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-2006-7125?page=106\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-2007-7130?page=109\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-2008-7134?page=107",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0238",
|
||||
"uid": "UID0238",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1982_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1982-7023?page=44",
|
||||
"source_split": "train"
|
||||
},
|
||||
{
|
||||
"id": "UID0241",
|
||||
"uid": "UID0241",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1963_06.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/june-1963-6798?page=13",
|
||||
"source_split": "train"
|
||||
}
|
||||
]
|
||||
194
data/officeqa_id_split/val/items.json
Normal file
194
data/officeqa_id_split/val/items.json
Normal file
@@ -0,0 +1,194 @@
|
||||
[
|
||||
{
|
||||
"id": "UID0001",
|
||||
"uid": "UID0001",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1941_01.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/january-1941-6529?page=15",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0027",
|
||||
"uid": "UID0027",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1970_06.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/june-1970-6882?page=89&deep=true",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0039",
|
||||
"uid": "UID0039",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_2004_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-2004-7117?page=20\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-2004-7117?page=21&deep=true",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0041",
|
||||
"uid": "UID0041",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1970_10.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/october-1970-6886?page=35",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0052",
|
||||
"uid": "UID0052",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_2000_06.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/june-2000-7102?page=56",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0070",
|
||||
"uid": "UID0070",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1939_01.txt\r\ntreasury_bulletin_1939_02.txt\r\ntreasury_bulletin_1939_03.txt\r\ntreasury_bulletin_1939_04.txt\r\ntreasury_bulletin_1939_05.txt\r\ntreasury_bulletin_1939_06.txt\r\ntreasury_bulletin_1939_07.txt\r\ntreasury_bulletin_1939_08.txt\r\ntreasury_bulletin_1939_09.txt\r\ntreasury_bulletin_1939_10.txt\r\ntreasury_bulletin_1939_11.txt\r\ntreasury_bulletin_1939_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/january-1939-6518?page=81\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/february-1939-6505?page=111\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1939-6519?page=117\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1939-6506?page=95\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/may-1939-6507?page=109\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/june-1939-6508?page=117\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/july-1939-6509?page=109\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/august-1939-6510?page=117\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/september-1939-6511?page=66&deep=true \r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/october-1939-6520?page=68\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/november-1939-6512?page=70\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/december-1939-6513?page=72",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0072",
|
||||
"uid": "UID0072",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_2011_12.txt\r\ntreasury_bulletin_2016_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2011-7149?page=58\r\n\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2016-535293?page=57",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0086",
|
||||
"uid": "UID0086",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_2022_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2022-627778?page=86",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0091",
|
||||
"uid": "UID0091",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1940_12.txt\r\ntreasury_bulletin_1941_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-1940-6528?page=21\r\n\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/december-1941-6540?page=64",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0109",
|
||||
"uid": "UID0109",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_2015_12.txt\r\ntreasury_bulletin_2020_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2015-519209?page=21\r\n \r\n https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-2020-598551?page=24",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0142",
|
||||
"uid": "UID0142",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1944_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1944-6567?page=93",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0154",
|
||||
"uid": "UID0154",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1977_03.txt\r\ntreasury_bulletin_1978_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1977-6963?page=83\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1978-6975?page=84",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0159",
|
||||
"uid": "UID0159",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_2000_09.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/september-2000-7103?page=109",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0161",
|
||||
"uid": "UID0161",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1980_03.txt\r\ntreasury_bulletin_1985_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1980-6999?page=88\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1985-7040?page=48",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0170",
|
||||
"uid": "UID0170",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1960_03.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/march-1960-6759?page=64",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0190",
|
||||
"uid": "UID0190",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1939_10.txt\r\ntreasury_bulletin_1939_11.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/october-1939-6520?page=14\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/november-1939-6512?page=14",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0213",
|
||||
"uid": "UID0213",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1947_04.txt\r\ntreasury_bulletin_1948_04.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1947-6603?page=28\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1948-6615?page=18",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0217",
|
||||
"uid": "UID0217",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1963_10.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/october-1963-6802?page=15",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0220",
|
||||
"uid": "UID0220",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1939_02.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/february-1939-6505?page=25",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0233",
|
||||
"uid": "UID0233",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1948_04.txt\r\ntreasury_bulletin_1958_04.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1948-6615?page=42\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1958-6735?page=54",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0234",
|
||||
"uid": "UID0234",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1958_01.txt\r\ntreasury_bulletin_1958_02.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/january-1958-6732?page=28\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/february-1958-6733?page=32",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0235",
|
||||
"uid": "UID0235",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1948_04.txt\r\ntreasury_bulletin_1948_05.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/april-1948-6615?page=27\r\nhttps://fraser.stlouisfed.org/title/treasury-bulletin-407/may-1948-6616?page=27",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0239",
|
||||
"uid": "UID0239",
|
||||
"category": "easy",
|
||||
"source_files": "treasury_bulletin_1953_01.txt\r\ntreasury_bulletin_1954_01.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/january-1953-6672?page=62\r\nhttp://fraser.stlouisfed.org/title/treasury-bulletin-407/january-1954-6684?page=51",
|
||||
"source_split": "val"
|
||||
},
|
||||
{
|
||||
"id": "UID0240",
|
||||
"uid": "UID0240",
|
||||
"category": "hard",
|
||||
"source_files": "treasury_bulletin_1957_12.txt",
|
||||
"source_docs": "https://fraser.stlouisfed.org/title/treasury-bulletin-407/december-1957-6731?page=26",
|
||||
"source_split": "val"
|
||||
}
|
||||
]
|
||||
21
data/searchqa_id_split/split_manifest.json
Normal file
21
data/searchqa_id_split/split_manifest.json
Normal file
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"benchmark": "SearchQA",
|
||||
"manifest_type": "id_split",
|
||||
"source_repo": "lucadiliello/searchqa",
|
||||
"source_repo_type": "dataset",
|
||||
"source_url": "https://huggingface.co/datasets/lucadiliello/searchqa",
|
||||
"source_id_field": "key",
|
||||
"counts": {
|
||||
"train": 400,
|
||||
"val": 200,
|
||||
"test": 1400
|
||||
},
|
||||
"item_fields": [
|
||||
"id"
|
||||
],
|
||||
"notes": [
|
||||
"This is a split manifest, not the full SearchQA payload.",
|
||||
"Materialize full split items from lucadiliello/searchqa before evaluation.",
|
||||
"The IDs in items.json exactly match the key field in lucadiliello/searchqa."
|
||||
]
|
||||
}
|
||||
24
data/spreadsheetbench_id_split/split_manifest.json
Normal file
24
data/spreadsheetbench_id_split/split_manifest.json
Normal file
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"benchmark": "SpreadsheetBench",
|
||||
"manifest_type": "id_split",
|
||||
"source_repo": "KAKA22/SpreadsheetBench",
|
||||
"source_repo_type": "dataset",
|
||||
"source_url": "https://huggingface.co/datasets/KAKA22/SpreadsheetBench",
|
||||
"source_revision": "ab0b742b0fc95b946f212d80ac7771b5531272e4",
|
||||
"source_file": "spreadsheetbench_verified_400.tar.gz",
|
||||
"source_split_name": "spreadsheetbench_split",
|
||||
"counts": {
|
||||
"train": 80,
|
||||
"val": 40,
|
||||
"test": 280
|
||||
},
|
||||
"item_fields": [
|
||||
"id",
|
||||
"spreadsheet_path",
|
||||
"instruction_type"
|
||||
],
|
||||
"notes": [
|
||||
"This is a split manifest, not the full SpreadsheetBench payload.",
|
||||
"Materialize full task JSON rows plus spreadsheet files from SpreadsheetBench Verified 400 before evaluation."
|
||||
]
|
||||
}
|
||||
1402
data/spreadsheetbench_id_split/test/items.json
Normal file
1402
data/spreadsheetbench_id_split/test/items.json
Normal file
File diff suppressed because it is too large
Load Diff
402
data/spreadsheetbench_id_split/train/items.json
Normal file
402
data/spreadsheetbench_id_split/train/items.json
Normal file
@@ -0,0 +1,402 @@
|
||||
[
|
||||
{
|
||||
"id": "32438",
|
||||
"spreadsheet_path": "spreadsheet/32438",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "398-14",
|
||||
"spreadsheet_path": "spreadsheet/398-14",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "47766",
|
||||
"spreadsheet_path": "spreadsheet/47766",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "48365",
|
||||
"spreadsheet_path": "spreadsheet/48365",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "32255",
|
||||
"spreadsheet_path": "spreadsheet/32255",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "10747",
|
||||
"spreadsheet_path": "spreadsheet/10747",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "50916",
|
||||
"spreadsheet_path": "spreadsheet/50916",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "577-40",
|
||||
"spreadsheet_path": "spreadsheet/577-40",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "35742",
|
||||
"spreadsheet_path": "spreadsheet/35742",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "46121",
|
||||
"spreadsheet_path": "spreadsheet/46121",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "51090",
|
||||
"spreadsheet_path": "spreadsheet/51090",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "51249",
|
||||
"spreadsheet_path": "spreadsheet/51249",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "82-30",
|
||||
"spreadsheet_path": "spreadsheet/82-30",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "56274",
|
||||
"spreadsheet_path": "spreadsheet/56274",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "57445",
|
||||
"spreadsheet_path": "spreadsheet/57445",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "46646",
|
||||
"spreadsheet_path": "spreadsheet/46646",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "105-24",
|
||||
"spreadsheet_path": "spreadsheet/105-24",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "6239",
|
||||
"spreadsheet_path": "spreadsheet/6239",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "414-20",
|
||||
"spreadsheet_path": "spreadsheet/414-20",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "165-23",
|
||||
"spreadsheet_path": "spreadsheet/165-23",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "40892",
|
||||
"spreadsheet_path": "spreadsheet/40892",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "48745",
|
||||
"spreadsheet_path": "spreadsheet/48745",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "32612",
|
||||
"spreadsheet_path": "spreadsheet/32612",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "325-44",
|
||||
"spreadsheet_path": "spreadsheet/325-44",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "262-17",
|
||||
"spreadsheet_path": "spreadsheet/262-17",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "141-20",
|
||||
"spreadsheet_path": "spreadsheet/141-20",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "52216",
|
||||
"spreadsheet_path": "spreadsheet/52216",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "22-47",
|
||||
"spreadsheet_path": "spreadsheet/22-47",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "55421",
|
||||
"spreadsheet_path": "spreadsheet/55421",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "56427",
|
||||
"spreadsheet_path": "spreadsheet/56427",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "36097",
|
||||
"spreadsheet_path": "spreadsheet/36097",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "32902",
|
||||
"spreadsheet_path": "spreadsheet/32902",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "32023",
|
||||
"spreadsheet_path": "spreadsheet/32023",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "1818",
|
||||
"spreadsheet_path": "spreadsheet/1818",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "170-13",
|
||||
"spreadsheet_path": "spreadsheet/170-13",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "66-24",
|
||||
"spreadsheet_path": "spreadsheet/66-24",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "58949",
|
||||
"spreadsheet_path": "spreadsheet/58949",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "42354",
|
||||
"spreadsheet_path": "spreadsheet/42354",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "194-19",
|
||||
"spreadsheet_path": "spreadsheet/194-19",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "31915",
|
||||
"spreadsheet_path": "spreadsheet/31915",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "58499",
|
||||
"spreadsheet_path": "spreadsheet/58499",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "45372",
|
||||
"spreadsheet_path": "spreadsheet/45372",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "11842",
|
||||
"spreadsheet_path": "spreadsheet/11842",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "57558",
|
||||
"spreadsheet_path": "spreadsheet/57558",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "472-15",
|
||||
"spreadsheet_path": "spreadsheet/472-15",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "55060",
|
||||
"spreadsheet_path": "spreadsheet/55060",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "31011",
|
||||
"spreadsheet_path": "spreadsheet/31011",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "408-39",
|
||||
"spreadsheet_path": "spreadsheet/408-39",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "54085",
|
||||
"spreadsheet_path": "spreadsheet/54085",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "39903",
|
||||
"spreadsheet_path": "spreadsheet/39903",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "48983",
|
||||
"spreadsheet_path": "spreadsheet/48983",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "108-24",
|
||||
"spreadsheet_path": "spreadsheet/108-24",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "58484",
|
||||
"spreadsheet_path": "spreadsheet/58484",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "118-50",
|
||||
"spreadsheet_path": "spreadsheet/118-50",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "10452",
|
||||
"spreadsheet_path": "spreadsheet/10452",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "39931",
|
||||
"spreadsheet_path": "spreadsheet/39931",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "3413",
|
||||
"spreadsheet_path": "spreadsheet/3413",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "247-24",
|
||||
"spreadsheet_path": "spreadsheet/247-24",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "56786",
|
||||
"spreadsheet_path": "spreadsheet/56786",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "55965",
|
||||
"spreadsheet_path": "spreadsheet/55965",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "379-36",
|
||||
"spreadsheet_path": "spreadsheet/379-36",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "58109",
|
||||
"spreadsheet_path": "spreadsheet/58109",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "433-47",
|
||||
"spreadsheet_path": "spreadsheet/433-47",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "192-22",
|
||||
"spreadsheet_path": "spreadsheet/192-22",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "49333",
|
||||
"spreadsheet_path": "spreadsheet/49333",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "493-18",
|
||||
"spreadsheet_path": "spreadsheet/493-18",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "54638",
|
||||
"spreadsheet_path": "spreadsheet/54638",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "34033",
|
||||
"spreadsheet_path": "spreadsheet/34033",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "30930",
|
||||
"spreadsheet_path": "spreadsheet/30930",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "585-41",
|
||||
"spreadsheet_path": "spreadsheet/585-41",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "32337",
|
||||
"spreadsheet_path": "spreadsheet/32337",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "55427",
|
||||
"spreadsheet_path": "spreadsheet/55427",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "263-1",
|
||||
"spreadsheet_path": "spreadsheet/263-1",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "254-34",
|
||||
"spreadsheet_path": "spreadsheet/254-34",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "57113",
|
||||
"spreadsheet_path": "spreadsheet/57113",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "57743",
|
||||
"spreadsheet_path": "spreadsheet/57743",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "43589",
|
||||
"spreadsheet_path": "spreadsheet/43589",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "250-20",
|
||||
"spreadsheet_path": "spreadsheet/250-20",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "48080",
|
||||
"spreadsheet_path": "spreadsheet/48080",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "370-43",
|
||||
"spreadsheet_path": "spreadsheet/370-43",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
}
|
||||
]
|
||||
202
data/spreadsheetbench_id_split/val/items.json
Normal file
202
data/spreadsheetbench_id_split/val/items.json
Normal file
@@ -0,0 +1,202 @@
|
||||
[
|
||||
{
|
||||
"id": "45635",
|
||||
"spreadsheet_path": "spreadsheet/45635",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "560-12",
|
||||
"spreadsheet_path": "spreadsheet/560-12",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "55049",
|
||||
"spreadsheet_path": "spreadsheet/55049",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "9569",
|
||||
"spreadsheet_path": "spreadsheet/9569",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "7902",
|
||||
"spreadsheet_path": "spreadsheet/7902",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "227-40",
|
||||
"spreadsheet_path": "spreadsheet/227-40",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "463-17",
|
||||
"spreadsheet_path": "spreadsheet/463-17",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "54144",
|
||||
"spreadsheet_path": "spreadsheet/54144",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "80-42",
|
||||
"spreadsheet_path": "spreadsheet/80-42",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "2768",
|
||||
"spreadsheet_path": "spreadsheet/2768",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "37456",
|
||||
"spreadsheet_path": "spreadsheet/37456",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "12864",
|
||||
"spreadsheet_path": "spreadsheet/12864",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "55979",
|
||||
"spreadsheet_path": "spreadsheet/55979",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "48620",
|
||||
"spreadsheet_path": "spreadsheet/48620",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "48588",
|
||||
"spreadsheet_path": "spreadsheet/48588",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "395-36",
|
||||
"spreadsheet_path": "spreadsheet/395-36",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "382-10",
|
||||
"spreadsheet_path": "spreadsheet/382-10",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "59595",
|
||||
"spreadsheet_path": "spreadsheet/59595",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "53383",
|
||||
"spreadsheet_path": "spreadsheet/53383",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "48921",
|
||||
"spreadsheet_path": "spreadsheet/48921",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "416-15",
|
||||
"spreadsheet_path": "spreadsheet/416-15",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "47798",
|
||||
"spreadsheet_path": "spreadsheet/47798",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "56563",
|
||||
"spreadsheet_path": "spreadsheet/56563",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "46897",
|
||||
"spreadsheet_path": "spreadsheet/46897",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "9726",
|
||||
"spreadsheet_path": "spreadsheet/9726",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "50768",
|
||||
"spreadsheet_path": "spreadsheet/50768",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "51-12",
|
||||
"spreadsheet_path": "spreadsheet/51-12",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "31628",
|
||||
"spreadsheet_path": "spreadsheet/31628",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "39046",
|
||||
"spreadsheet_path": "spreadsheet/39046",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "8942",
|
||||
"spreadsheet_path": "spreadsheet/8942",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "48527",
|
||||
"spreadsheet_path": "spreadsheet/48527",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "59196",
|
||||
"spreadsheet_path": "spreadsheet/59196",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "6698",
|
||||
"spreadsheet_path": "spreadsheet/6698",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "43436",
|
||||
"spreadsheet_path": "spreadsheet/43436",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "38462",
|
||||
"spreadsheet_path": "spreadsheet/38462",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "402-43",
|
||||
"spreadsheet_path": "spreadsheet/402-43",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "267-18",
|
||||
"spreadsheet_path": "spreadsheet/267-18",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "37378",
|
||||
"spreadsheet_path": "spreadsheet/37378",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "53647",
|
||||
"spreadsheet_path": "spreadsheet/53647",
|
||||
"instruction_type": "Cell-Level Manipulation"
|
||||
},
|
||||
{
|
||||
"id": "142-12",
|
||||
"spreadsheet_path": "spreadsheet/142-12",
|
||||
"instruction_type": "Sheet-Level Manipulation"
|
||||
}
|
||||
]
|
||||
Reference in New Issue
Block a user