Nitish Joshi

Recent News

• (8/2020) Started as a PhD student at NYU Courant working in the ML2 group
• (4/2020) I've been awarded the National Science Foundation Graduate Research Fellowship!
• (1/2020) Paper on accent adaptation for speech recognition accepted to ICASSP 2020
• (7/2019) Presented two papers on Multi-hop Reading Comprehension and Cross-lingual question generation at ACL 2019 in Florence, Italy

Publications

2020

• Coupled Training of Sequence-to-Sequence Models for Accented Speech Recognition
  Vinit Unni*, Nitish Joshi*, and Preethi Jyothi.
  ICASSP 2020
  [bib] [abstract]
  
  

  @inproceedings{UnniJoshi2020coupled,
    author={Vinit Unni, Nitish Joshi, and Preethi Jyothi},
    booktitle={ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, 
    title={Coupled Training of Sequence-to-Sequence Models for Accented Speech Recognition}, 
    year={2020},
  }
  
                              

  
  

  Accented speech poses significant challenges for state-of-the-art automatic speech recognition (ASR) systems. Accent is a property of speech that lasts throughout an utterance in varying degrees of strength. This makes it hard to isolate the influence of accent on individual speech sounds. We propose coupled training for encoder-decoder ASR models that acts on pairs of utterances corresponding to the same text spoken by speakers with different accents. This training regime introduces an L2 loss between the attention-weighted representations corresponding to pairs of utterances with the same text, thus acting as a regularizer and encouraging representations from the encoder to be more accent-invariant. We focus on recognizing accented English samples from the Mozilla Common Voice corpus. We obtain significant error rate reductions on accented samples from a large set of diverse accents using coupled training. We also show consistent improvements in performance on heavily accented samples (as determined by a standalone accent classifier).

2019

• Explore, Propose and Assemble: An Interpretable Model for Multi-hop Reading Comprehension
  Yichen Jiang*, Nitish Joshi*, Yen-Chun Chen and Mohit Bansal
  ACL 2019
  [bib] [abstract] [code]
  
  

  @inproceedings{JiangJoshi2019epar, 
      author={Yichen Jiang, Nitish Joshi, Yen-chun Chen and Mohit Bansal}, 
      booktitle={Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics}, 
      title={Explore, Propose, and Assemble: An Interpretable Model for Multi-Hop Reading Comprehension}, 
      year={2019}
  }
                              

  
  

  Multi-hop reading comprehension requires the model to explore and connect relevant information from multiple sentences/documents in order to answer the question about the context. To achieve this, we propose an interpretable 3-module system called Explore-Propose-Assemble reader (EPAr). First, the Document Explorer iteratively selects relevant documents and represents divergent reasoning chains in a tree structure so as to allow assimilating information from all chains. The Answer Proposer then proposes an answer from every root-to-leaf path in the reasoning tree. Finally, the Evidence Assembler extracts a key sentence containing the proposed answer from every path and combines them to predict the final answer. Intuitively, EPAr approximates the coarse-to-fine-grained comprehension behavior of human readers when facing multiple long documents. We jointly optimize our 3 modules by minimizing the sum of losses from each stage conditioned on the previous stage’s output. On two multi-hop reading comprehension datasets WikiHop and MedHop, our EPAr model achieves significant improvements over the baseline and competitive results compared to the state-of-the-art model. We also present multiple reasoning-chain-recovery tests and ablation studies to demonstrate our system’s ability to perform interpretable and accurate reasoning.



• Cross-lingual Training for Automatic Question Generation
  Vishwajeet Kumar, Nitish Joshi, Arijit Mukherjee, Ganesh Ramakrishnan, and Preethi Jyothi.
  ACL 2019
  [bib] [abstract] [dataset]
  
  

  @inproceedings{kumar-etal-2019-cross,
      author={Vishwajeet Kumar, Nitish Joshi, Arijit Mukherjee, Ganesh Ramakrishnan and Preethi Jyothi},
      booktitle={Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
      title={Cross-Lingual Training for Automatic Question Generation},
      year={2019}
  }
                              

  
  

  Automatic question generation (QG) is a challenging problem in natural language understanding. QG systems are typically built assuming access to a large number of training instances where each instance is a question and its corresponding answer. For a new language, such training instances are hard to obtain making the QG problem even more challenging. Using this as our motivation, we study the reuse of an available large QG dataset in a secondary language (e.g. English) to learn a QG model for a primary language (e.g. Hindi) of interest. For the primary language, we assume access to a large amount of monolingual text but only a small QG dataset. We propose a cross-lingual QG model which uses the following training regime: (i) Unsupervised pretraining of language models in both primary and secondary languages and (ii) joint supervised training for QG in both languages. We demonstrate the efficacy of our proposed approach using two different primary languages, Hindi and Chinese. Our proposed framework clearly outperforms a number of baseline models, including a fully-supervised transformer-based model trained on the QG datasets in the primary language. We also create and release a new question answering dataset for Hindi consisting of 6555 sentences.

Miscellany

• In my free time, I like to go for a run or cook. I also like bird watching and photography.
• The source code for this website was borrowed from Nelson Liu (https://nelsonliu.me)