I am a postdoctoral researcher at the College of Information and Computer Sciences at UMass Amherst.
I work in the Information Extraction and Synthesis Laboratory under the direction of Andrew McCallum.
I am currently completing a Postdoc in Computer Science under the direction of Andrew McCallum with a focus on structured representation learning. I obtained my Ph.D. in Mathematics from UMass Amherst in 2018, advised by Andrea Nahmod and Nestor Guillen. My Ph.D. research was in the field of partial differential equations, using techniques from harmonic analysis, calculus of variations, and Riemannian geometry. At the same time, I also completed my MS in Computer Science at UMass Amherst, with a focus on machine learning. Prior to my graduate studies, I obtained a BA in Mathematics and Economics (double-major) from Central Connecticut State University in 2010, graduating in two years with a 4.0 GPA.
I perform research on foundational aspects of machine learning, particularly representation learning and probabilistic modeling, with applications to natural language processing and structured prediction. I am particularly interested in models with additional geometric, probabilistic, or set-theoretic structure, and the way in which this structure can be leveraged to offer additional capabilities and interpretability to deep learning models. The prime example of such a representation is box embeddings, a novel region-based representation learning model which also compactly represents a joint probability distribution.
To this end, over the course of 10 research papers, I have provided methodological improvements and extensions to box embeddings, as well as applying them to a wide range of tasks such as collaborative filtering, textual entailment, and multi-label classification, obtaining state-of-the-art results. Box embeddings represent elements as hyperrectangles, i.e. Cartesian products of intervals, and can be thought of as trainable Venn-diagrams with valid set-theoretic and probabilistic semantics. I formalized the probabilistic semantics of box embeddings (UAI 2021), proving that even models with softness can yield a valid probability distribution and, as a result, outperform competing probabilistic representation baselines. I also proved that box embeddings can represent any directed graph (NeurIPS 2021) and introduce a novel adaptation of box embeddings with a trainable "softness", improving learning to the point that they are the optimal choice for directed graph representation in any dimension.
My current research formalizes the set-theoretic aspects of box representations by casting them in a general framework, building from first principles the general requirements for any set-theoretic representation learning model. With this rigorous framework in hand, I plan to extend the expressivity of box embeddings in multiple ways - exploring alternate geometries (such as tori or hyperbolic space), combining boxes with probabilistic circuits, and by developing deep box models capable of rich, probabilistically interpretable reasoning in their hidden layers. Developing this abstract framework and requirements for set-theoretic representation learning also lays bare the foundational principles on which such a representation should be built, facilitating analysis of alternative representations which use geometric, region-based or distributional representations, and providing sound motivation for the invention of novel representation paradigms. As set-theoretic representation and reasoning is so foundational, not only to mathematics and probability but also in the organization of ideas into concepts and general logic of thought,
While vectors in Euclidean space have formed the basis of most machine learning architectures, several promising lines of work are exploring the extent to which objects with additional geometric structure may provide distinct benefits. Vectors in hyperbolic space, for example, can model trees with lower distortion, an idea which has been extended to Riemannian and Lorentzian manifolds wherein the model learns the curvature best suited to the data. Gaussian embeddings, on the other hand, learn representations of the data which are themselves distributions, allowing the representation to learn some notion of uncertainty and also providing natural asymmetric measures (eg. KL divergence) between the representations. Finally, region-based embeddings represent elements using regions such as cones, disks, and boxes, allowing access to a rich set of set-theoretic relations such as intersection, complement, and containment. With an appropriate measure on the embedding space, one can also calculate volumes of these regions, allowing the embeddings to be interpreted with rigorous probabilistic semantics. Of these various choices of region-based representations, box embeddings stand out due to their representational capacity, tractable computability, and simplicity of implementation.
More broadly, I am interested in program synthesis, deep learning, optimization techniques, and interpretability. I am also interested in game theory, parallel and distributed algorithms, programming languages and theory of computation. I enjoy exploring the theoretical underpinnings of machine learning and identifying areas where my mathematical background can be leveraged to improve performance and solve real problems of practical importance with a large impact.
As mentioned above, for my mathematics Ph.D. I studied partial differential equations using techniques from harmonic analysis, calculus of variations, and Riemannian geometry. My thesis is comprised of two parts, the first of which improves bounds on the Sobolev norms of solutions to the Nonlinear Schrödinger equation in dimensions 2, 3, and 4. In the second part I proved a uniqueness theorem for solutions to a class of degenerate elliptic partial differential equations. The thesis is published online, and can be accessed from ScholarWorks:
During high school I started a website development and IT consulting company called Starstreak, which allowed me to get experience with a wide range of software and hardware, from niche industries to the enterprise level. Running the business also allowed me to gain a broad exposure to a wide range of clientele, many of whom became my friends over the years.
I also have a love for music, and am fortunate to have played trumpet and piano and even sing with very talented musicians in classical, jazz, and rock bands. At the moment I sing and arrange music for a horn band which takes inspiration (and occasionally direct transcriptions!) from artists such as Lawrence and Cory Wong to perform everything from modern pop like Maroon 5 to classic funk and soul like Tower of Power.