Elias Hossain

R-GAT: Residual Graph Attention Network for Biomedical Document Classification

This completed project proposes R-GAT, a Residual Graph Attention Network for classifying medical documents from the PubMed database. It encodes textual units as graph nodes and semantic relations as edges, effectively handling long-form biomedical texts across thyroid, colon, lung, and generic categories.

The model computes node representations using multi-head graph attention:

\[ h_i' = \sigma\left(\sum_{j \in \mathcal{N}(i)} \alpha_{ij} W h_j\right) \]

\[ \alpha_{ij} = \frac{\exp\left(\text{LeakyReLU}(a^T [Wh_i \, || \, Wh_j])\right)}{\sum_{k \in \mathcal{N}(i)} \exp\left(\text{LeakyReLU}(a^T [Wh_i \, || \, Wh_k])\right)} \]

Residual connections and global pooling improve training stability and classification performance. A version has been submitted as a preprint to arXiv.

[Read the preprint on arXiv]

LLM Reasoning: Enhancing Multi-Step Reasoning in Large Language Models

This ongoing project investigates the reasoning capabilities of Large Language Models (LLMs) in multi-hop and chain-of-thought tasks. We explore methods to inject structured reasoning using step supervision and logical consistency constraints.

We define a hybrid loss combining language modeling with reasoning consistency:

\[ \mathcal{L}_{\text{total}} = \lambda_1 \cdot \mathcal{L}_{\text{LM}} + \lambda_2 \cdot \mathcal{L}_{\text{reason}} \]

Here, \( \mathcal{L}_{\text{LM}} \) is the standard token prediction loss, and \( \mathcal{L}_{\text{reason}} \) ensures intermediate steps follow logical structure. We evaluate on datasets like GSM8K, MEDQA, and StrategyQA.

📄 Preprint coming soon

Contrastive Learning in Positive-Unlabeled Data with LLMs

This project explores the integration of contrastive learning with Large Language Models (LLMs) to enhance classification in Positive-Unlabeled (PU) settings, where only partial labels are available. The method learns discriminative representations by contrasting positive examples against ambiguous unlabeled data in embedding space.

We fine-tune a pretrained LLM using a contrastive loss that separates positive pairs from uncertain examples while preserving semantic alignment:

\[ \mathcal{L}_{\text{total}} = \lambda_1 \cdot \mathcal{L}_{\text{LM}} + \lambda_2 \cdot \mathcal{L}_{\text{cont}} \]

\[ \mathcal{L}_{\text{cont}} = -\log \frac{\exp(\text{sim}(z_i, z_j)/\tau)}{\sum_{k=1}^{K} \exp(\text{sim}(z_i, z_k)/\tau)} \]

Here, \( \text{sim}(z_i, z_j) \) denotes cosine similarity between representations, and \( \tau \) is a temperature parameter. This framework helps the LLM distinguish informative unlabeled instances without requiring full supervision.

📄 Manuscript in progress