# IQA Dataset

This is a simple code to generate IQA dataset based on Facebook bAbI. Currently, we focus on three tasks, including task 1, task 4 and task 7. There are two folders:

  - `gen_iqa_ibAbI_tasks`: generates ibAbI datasets for three tasks.
  - `iqa_sample_data`: prodives some sample ibAbI datasets for three tasks.

## IQA Dataset Generation
The code is under folder `gen_iqa_ibAbI_tasks`, which is implemented based on Facebook bAbI. Its installation and usage are as the same as Facebook bAbI.

### Installation
 To use the code, install it with LuaRocks by running the following command from the root directory.
 
 ```sh
luarocks make babitasks-scm-1.rockspec
```

### Usage
To generate a task, run the command

 ```sh
babi-tasks <task-id>
```

where <task-id> is either a class name (like WhereIsActor) or the task number (e.g. 1). To quickly generate 1k examples and store in a file, you can use

 ```sh
babi-tasks <task-id> 10000 > (path/)filename
```

Please note that we currently support three tasks for IQA, including task 1, task 4, and task7.

## IQA Sample Dataset

We provide some sample datasets for task 1, task 4, and task 7 under folder `iqa_sample_data`. Under each task, we have a list of sub-folders, such as,

- r_1.0
- r_0.9
- ...
- raw

where `raw` contains both training and testing data generated by the code. `r_<R_IQA>` represents that IQA mixture ratio is `R_IQA`. Specifically, we randomly pick `R_IQA` data from ibAbI data, and then select the remaining data from bAbI data with the corresponding task. In each `r_<R_IQA>`, there are three folders which provide datasets in different formats,

- `mix`: which is the mixture output of both IQA problems and regular QA problems.
- `regular`: which removes the supplementary questions and feedbacks upon `mix` output, and is used as inputs for regular QA models. 
- `formatted`: which is only used for our own usage.

## Data Format
This is an example for IQA problem.

```sh
1 John travelled to the hallway.
2 Daniel went to the bedroom.
3 Where is he?	bedroom	2
4 SQ: who is he?	Daniel	2
```

Line 1 and Line 2 are input statements. There are three parts in Line 3. The first part `Where is he?` is target question. The second part `bedroom` is  final answer. The last one is supporting number. There are also three parts in Line 4. The first part `SQ: who is he?` is supplementary question, and the second part `Daniel` is the feedback to this supplementary question. The last number is the supporting number. The above example is equivalent to the following example which is more readable and more easier to follow,

```sh
1 Input statements: John travelled to the hallway.
2 Input statements: Daniel went to the bedroom.
3 Question: Where is he?
4 Supplementary Question: who is he?
5 Feedback: Daniel
6 Answer: bedroom
```