VSE++: Improved Visual-Semantic Embeddings
Fartash Faghri, David J Fleet, Jamie Ryan Kiros, Sanja Fidler. Arxiv, 2017.
[arXiv] [code]

Simple modifications that lead to state of the art image-sentence retrieval on MSCOCO.

Joint Embeddings of Scene Graphs and Images
Eugene Belilovsky, Matthew Blaschko, Jamie Ryan Kiros, Raquel Urtasun, Richard Zemel. ICLR Workshop, 2017.
[OpenReview]

A collection of baselines for embedding scene graphs and images.

Layer Normalization
Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton. Arxiv, 2016.
[arXiv] [code]

A normalization technique for speeding up RNN training.

Towards Generalizable Sentence Embeddings
Eleni Triantafillou, Jamie Ryan Kiros, Raquel Urtasun, Richard Zemel. ACL Workshop on Representation Learning for NLP, 2016.
[pdf]

Exploiting a small amount of supervision for unsupervised sentence representations.

Order-Embeddings of Images and Language
Ivan Vendrov, Jamie Ryan Kiros, Sanja Fidler, Raquel Urtasun. ICLR, 2016. (Oral)
[arXiv] [code]

A general method for learning ordered representations applied to tasks involving images and language.

Action Recognition using Visual Attention
Shikhar Sharma, Jamie Ryan Kiros, Ruslan Salakhutdinov. NIPS Time Series Workshop, 2015.
[arXiv] [project page]

A first attempt at using soft visual attention for action recognition.

Exploratory Visual Analysis and Interactive Pattern Extraction from Semi-Structured Data
Axel J. Soto, Jamie Ryan Kiros, Vlado Keselj, Evangelos Milios. ACM Transactions on Interactive and Intelligent Systems, 2015.
[link]

Introducing ViTA-SSD, a visual text analytics tool for semi-structured data that utilizes a new deep learning algorithm for visualization.

Skip-Thought Vectors
Jamie Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, Sanja Fidler. NIPS, 2015.
[arXiv] [code]

How to learn a generic, off-the-shelf sentence encoder.

Exploring Models and Data for Image Question Answering
Mengye Ren, Jamie Ryan Kiros, Richard Zemel. NIPS, 2015.
[arXiv] [code] [full results] [COCO-QA Dataset]

Combining CNNs with LSTM for image question answering and a new dataset derived from Microsoft COCO captions.

Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
Yukun Zhu, Jamie Ryan Kiros, Richard Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, Sanja Fidler. ICCV, 2015. (Oral)
[arXiv] [project page]

Aligning movies and books for story-like captioning.

Scalable Bayesian Optimization Using Deep Neural Networks
Jasper Snoek, Oren Rippel, Kevin Swersky, Jamie Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Md. Mostofa Ali Patwary, Prabhat, Ryan P. Adams. ICML, 2015.
[arXiv]

Replacing a Gaussian process with a neural network for Bayesian optimization with large-scale parallelism.

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Kelvin Xu, Jimmy Ba, Jamie Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, Yoshua Bengio. ICML, 2015.
[arXiv] [project page] [media]

An interpretable model for generating image descriptions.

Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
Jamie Ryan Kiros, Ruslan Salakhutdinov, Richard Zemel. TACL, 2015. (to appear)
[arXiv] [demo] [INTERESTING.JPG] [code]

An encoder-decoder architecture for ranking and generating image descriptions.
(Twitter account INTERESTING.JPG is authored and maintained by link, independent of us)

A Multiplicative Model for Learning Distributed Text-Based Attribute Representations
Jamie Ryan Kiros, Richard Zemel, Ruslan Salakhutdinov. NIPS, 2014.
[arXiv (old)]

A third order model for jointly learning distributed representations of words and document attributes.

Comparing algorithms for automated vessel segmentation in computed tomography scans of the lung: The VESSEL12 study
Rina D. Rudyanto et al. Medical Image Analysis, 2014.
[link]

The VESSEL12 segmentation study that subsumes our 2013 technical report below.

Multimodal Neural Language Models
Jamie Ryan Kiros, Ruslan Salakhutdinov, Richard Zemel. ICML, 2014.
[pdf] [project page and results]

Two new neural language models that can be conditioned on other modalities (such as images).

Learning Deep Representations, Embeddings and Codes from the Pixel Level of Natural and Medical Images
Jamie Ryan Kiros. Master's Thesis, University of Alberta, 2013.
[link]

MSc Outstanding Thesis Award.

Training Neural Networks with Stochastic Hessian-Free Optimization
Jamie Ryan Kiros. ICLR, 2013.
[pdf] [code] [openreview discussion]

A stochastic version of Martens' Hessian-Free Optimization for training deep nets.

Automatic Lung Vessel Segmentation via Stacked Multiscale Feature Learning
Jamie Ryan Kiros, Karteek Popuri, Matthew Low, Dana Cobzas, Martin Jagersand. Technical Report, 2013.
[pdf] [code] [competition page] [video] [results viewer]

1st place (out of 24) in the VESSEL12 segmentation challenge.

Deep Representations and Codes for Image Auto-Annotation
Jamie Ryan Kiros and Csaba Szepesvari. NIPS, 2012.
[pdf] [supplementary] [code and features]

Learning binary codes for image annotation, using a deep unsupervised net.

Regularizers Versus Losses for Nonlinear Dimensionality Reduction
Yaoliang Yu, James Neufeld, Jamie Ryan Kiros, Xinhua Zhang, Dale Schuurmans. ICML, 2012.
[pdf] [supplementary]

A unifying view of non-parametric, nonlinear dimensionality reduction + 2 new convex regularizers.

On Linear Embeddings and Unsupervised Feature Learning
Jamie Ryan Kiros and Csaba Szepesvari. ICML Representation Learning Workshop, 2012. (Oral)
[pdf] [code]

Combining k-means feature learning with supervised linear t-distributed embeddings.

Representation Learning for Sparse, High Dimensional Multi-Label Classification
Jamie Ryan Kiros, Axel J. Soto, Evangelos Milios, Vlado Keselj. RSCTC, 2012.
[pdf] [competition page]

5th place (out of 126) in the JRS 2012 data mining competition.