favorite16The approach that we take to probe for compositional meaning information in sentence embeddings is inspired by the neuroscience technique of multivariate pattern analysis (Haxby et al., 2014), which tests for encoding of information in patterns of neural data by means of classification tasks designed to be contingent on the information of interest.
favorite03) We make available the classification datasets used for these experiments, as well as the generation system used to produce the sentence sets, to allow for broader testing of composition models and to facilitate creation of new tasks and classification datasets.
favorite01) We introduce a method for analyzing compositional meaning information in sentence embeddings, along with a generation system that enables controlled creation of datasets for this analysis.
favorite8First, to create controlled datasets at the necessary scale, we develop a generation system that allows us to produce large sentence sets meeting specified semantic, syntactic and lexical constraints, with gold-standard meaning annotation for each sentence.
favorite5We present a method to address this challenge, developing tasks that directly target compositional meaning information in sentence vector representations with a high degree of precision and control.
favorite95Similar to the original A* algorithm on graphs [Hart et al., 1968], we need a heuristic function H to estimate the final weight continuing from the current search state (an instantiation in this case) to the closest goal item.
favorite4The generalized A* search algorithm from Felzenszwalb and McAllester  can then be applied with a monotonic and admissible heuristic function to find the k instantiations of the goal item with smallest weights, from which we get the k shortest paths.
favorite10Following Allauzen and Riley , we limit our effort in finding k shortest paths to WPDAs with a bounded stack in both pushing and popping.2 Definition 4.
favorite77In this paper, we introduce two efficient algorithms for finding the k shortest paths of a WPDA, both derived from the same weighted deductive logic description of the execution of a WPDA using different search strategies..
favorite2Since the WPDA expansion has an exponential time and space complexity with respect to the size of the automaton, one usually has to prune the WPDA before expansion (the pruned expansion approach), i.e. remove those transitions and states that are not on any accepting path with a weight at most a given threshold greater than the shortest distance.
favorite3As stated earlier (Sections 3.2 and 3.4), we use the entire training set to extract the candidate lists for concept prediction and relation prediction, but train our learning algorithm on only a subset of the sentence-AMR pairs in the training data, which is obtained by selecting sentences having less than a fixed number of spans (C, set to 10 for all our experiments).
favorite1To prune the search space of our learning task, and to improve the quality of predictions, we use two observations about the nature of the edges of the AMR of a sentence, and its dependency tree, within our algorithm.
favorite73 Methodology 3.1 Learning technique Algorithm 1 1: for each span si do 2: ci = predict concept(si ) 3: end for 4: croot = predict root([c1 , ..., cn ]) 5: for each concept ci do 6: for each j < i do 7: r(i,j) = predict relation(ci , cj ) 8: r(j,i) = predict relation(cj , ci ) 9: end for 10: end for We use SEARN as described in section 2 to learn a model that can successfully predict the AMR y for a sentence x.
favorite4(b) Sample current state for re- (c) Three possible actions given the current state for relation prediction, the last one being the true relation i.e. Figure 2: Using SEARN for AMR parsing to model the learning of concepts and relations in a unified framework which aims to minimize the loss over the entire predicted structure, as opposed to minimizing the loss over concepts and relations in two separate stages, as is done by Flanigan et al (2014).
favorite1We develop a novel technique to parse English sentences into Abstract Meaning Representation (AMR) using SEARN, a Learning to Search approach, by modeling the concept and the relation learning in a unified framework.