Page Not Found
Page not found. Your pixels are in another canvas.
Sitemap
A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.
Pages
Page not in menu
This is a page not in th emain menu
Posts
Perspectives on the future of AI
Published:
How big are the models going to get and how much longer is the scaling hypothesis going to hold? It’s unclear, but according to current performance trends, which haven’t shown signs of plateauing (GPT-4o, Claude 3.5 Sonnet, Gemini-1.5-Pro, Llama-3.1-405B, Grok-2), and the power budget of announced data centres (5GW OpenAI/Microsoft Stargate campus), it is likely that there is an order of magnitude left (OOM) to climb in model size. This Epoch AI research covers these scenarios in depth and estimates training runs of the order of ~2e29 FLOPs being possible by 2030, which would be 4 OOMs larger than GPT-4 (2e25 FLOPs). These training runs will primarily be power constrained, followed by chips, data, and latency.
Understanding Hansen’s global warming in the pipeline
Published:
In progress
A time capsule
Published:
In progress
The need for a critical mineral demand model incorporating technical change
Published:
Introduction
Studying the effects of technical change on critical mineral demand and supply in the context of the low-carbon energy transition is an important and open area of research. Despite the crucial role played by these minerals in low-carbon technologies, long-term demand projections remain uncertain due to intricate interactions between drivers of technical change. In this writeup, I lay out what a framework that studies the effects of technical change on critical mineral demand would look like, how it can be developed, and what are its potential use cases.
Developments in machine learning for antibody design
Published:
Protein structure and sequence modeling has seen a fresh wave of resurgence in the last couple of years owing to some interesting developments in machine learning (ML) and deep learning (DL) based techniques. These techniques appear in a variety of flavours including using Equivariant neural network modules to respect the structural properties of 3D macromolecules, deeper networks that can benefit from the increased available experimental structures, powerful node-to-node relationship learners like transformers, and masked language modeling on the protein sequence space to learn evolutionary information. While structure prediction methods like AlphaFold (AF) [1] and RosettaFold (RF) [2] have become ubiquitious in computational structural biology, there remain challenges to be tackled on multiple fronts, where ML will play an important role.
Are we explorers or caretakers?
Published:
This was written when I was younger, and both the content and the form of my opinions on this topic have changed since then. Leaving this here for the sake of continuity.
phd
portfolio
Portfolio item number 1
Short description of portfolio item number 1
Portfolio item number 2
Short description of portfolio item number 2
projects
Hybrid Precoding for mmWave Massive MIMO OFDM
Hybrid Precoding, Partially Connected Structure, 2018
Hybrid precoding, a combination of digital and analog precoding, is an alternative to traditional precoding methods in massive MIMO systems with a large number of antenna elements and has shown promising results recently. In this paper, we implement a parallel framework to make hybrid precoding competitive in fast-fading environments. A low-complexity algorithm which exploits the block diagonal phase-only nature of the analog precoder in a partially connected structure is proposed to arrive at a hybrid precoding solution for a multi-carrier single-user system using orthogonal frequency division multiplexing (OFDM). The original problem is broken down into two subproblems of finding the magnitude and the phase components which are solved independently. A per-RF chain power constraint is introduced instead of the sum power constraint over all antennas which are much more practical in real systems. An alternating version of the same algorithm is proposed for increased spectral-efficiency gains. Complexity and run-time analysis demonstrate the advantage of the proposed algorithm over existing hybrid precoding schemes for partially connected structure in an OFDM setting. The simulation results reveal certain insights about the partially connected structure and the tradeoffs that have to be made to make it workable in a real wideband system.
Character Based Language Models Through Variational Sentence and Word Embeddings
NLP, Language Model, 2018
Language models have come of age recently with the introduction of Long-Short-Term-Memory based encoders, decoders and the advent of the attention mechanism. These models however work by generating one word at a time and cannot account for character level similarities and differences. In this project we propose a novel character based hierarchical variational autoencoder framework that can learn the word and sentence embeddings at the same time. We couple this with an attention mechanism over the latent word embeddings to realize the end-to-end autoencoder framework.
Private machine learning: Extension to the bayesian setting and higher dimensional querying
Private Machine Learning, Bayesian, High Dimensional, 2018
We extended the private machine learning framework to include higher dimensional querying arriving at upper and lower bounds on sample complexity. Provided a primer for querying under the bayesian setting with a prior on the underlying distribution of keys.
Representation Learning Strategies for the Epigenome and Chromatin Structure using Recurrent Neural Models
Thesis, Thesis, 2023
In this Ph.D. thesis, we propose frameworks for designing informative position-specific representations from epigenomic and structural genomic signals. We use recurrent priors in our analysis owing to the fact that the genome is heavily correlated with nearby positions, and implement them using recurrent neural models. We demonstrate that the representations we learn are helpful for various tasks, including, locating known genomic elements, identifying conserved sites, correlating with established genomic measures, enabling accurate decoding, finding elements that drive 3D conformation, attributing relative positional importance, and performing in-silico modifications. In the process of designing these representations, we study two classes of strategies that differ in their underlying philosophy, namely, autoencoding and categorical encoding. We show that the usefulness of these representations depends on the underlying strategies used while designing them.
publications
A Downscaled Faster-RCNN Framework for Signal Detection and Time-Frequency Localization in Wideband RF Systems
Published in IEEE Transactions on Wireless Communications, 2020
We propose a wideband spectrum sensing technique to detect and localize wireless radio frequency (RF) signals of interest in time and frequency when uninteresting signals cause RF interference (RFI). Specifically, we adopt and downscale the existing Faster-RCNN (FRCNN) framework to achieve better signal detection and localization than the state-of-the-art. For experimental evaluation, we present a data generation framework for Wi-Fi as the signals of interest and the Bluetooth and microwave oven signals as the RFI. Experiments reveal that (i) the downscaled FRCNN model can achieve up to a mean average precision (mAP) of 0.8, significantly outperforming the state-of-the-art, (ii) feature extraction with the VGG-13 architecture gives the best mAP with pretrained weights and configured as trainable, (iii) for signal detection in real RF traces, when compared to training purely with synthetic RF data, a better mAP can be achieved by training with a mixture of synthetic and real RF traces or by fine tuning the synthetically-trained weights with an additional round of training on a small amount of real RF traces, and (iv) the mAP performance decreases as the signal to noise ratio (SNR) is lowered.
Recommended citation: Prasad, K. S. V., D’souza, K. B., & Bhargava, V. K. (2020). A downscaled faster-RCNN framework for signal detection and time-frequency localization in wideband RF systems. IEEE Transactions on Wireless Communications, 19(7), 4847-4862. Full Document
Latent representation of the human pan-celltype epigenome through a deep recurrent neural network
Published in IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2021
The availability of thousands of assays of epigenetic activity necessitates compressed representations of these data sets that summarize the epigenetic landscape of the genome. Until recently, most such representations were cell type-specific, applying to a single tissue or cell state. Recently, neural networks have made it possible to summarize data across tissues to produce a pan-cell type representation. In this work, we propose Epi-LSTM, a deep long short-term memory (LSTM) recurrent neural network autoencoder to capture the long-term dependencies in the epigenomic data. The latent representations from Epi-LSTM capture a variety of genomic phenomena, including gene-expression, promoter-enhancer interactions, replication timing, frequently interacting regions, and evolutionary conservation. These representations outperform existing methods in a majority of cell types, while yielding smoother representations along the genomic axis due to their sequential nature.
Recommended citation: Dsouza, K. B., Li, A. Y., Bhargava, V., & Libbrecht, M. W. (2021). Latent representation of the human pan-celltype epigenome through a deep recurrent neural network. IEEE/ACM Transactions on Computational Biology and Bioinformatics. Full Document
Wireless threat detection device, system, and methods to detect signals in wideband RF systems and localize related time and frequency information based on deep learning
Published in US Patent, 2021
The present invention comprises a novel system and method to detect and estimate the time-frequency span of wireless signals present in a wideband RF spectrum. In preferred embodiments, the Faster RCNN deep learning architecture is used to detect the presence of wireless transmitters from the spectrogram images plotted by searching for rectangular shapes of any size, then localize the time and frequency information from the output of the FRCNN deep learning architecture.
Recommended citation: Koppisetti, N. R. S. V. P., Dsouza, K. B., Boostanimehr, H., & Mallick, S. (2022). U.S. Patent Application No. 17/825,304. Full Document
Learning representations of chromatin contacts using a recurrent neural network identifies genomic drivers of conformation
Published in Nature Communications, 2021
Despite the availability of chromatin conformation capture experiments, discerning the relationship between the 1D genome and 3D conformation remains a challenge, which limits our understanding of their affect on gene expression and disease. We propose Hi-C-LSTM, a method that produces low-dimensional latent representations that summarize intra-chromosomal Hi-C contacts via a recurrent long short-term memory neural network model. We find that these representations contain all the information needed to recreate the observed Hi-C matrix with high accuracy, outperforming existing methods. These representations enable the identification of a variety of conformation-defining genomic elements, including nuclear compartments and conformation-related transcription factors. They furthermore enable in-silico perturbation experiments that measure the influence of cis-regulatory elements on conformation.
Recommended citation: Dsouza, K. B., Maslova, A., Al-Jibury, E., Merkenschlager, M., Bhargava, V. K., & Libbrecht, M. W. (2022). Learning representations of chromatin contacts using a recurrent neural network identifies genomic drivers of conformation. Nature Communications, 13(1), 1-19. Full Document
Assessing the climate benefits of afforestation: phenomenology, processes, methods
Published:
Afforestation greatly influences several earth system processes, making it essential to understand these effects to accurately assess its potential for climate change mitigation. Although our understanding of forest-climate interactions has improved, significant knowledge gaps remain, preventing definitive assessments of afforestation’s net climate benefits. In this review, focusing on the Canadian boreal, we identify these gaps and synthesize existing knowledge. The review highlights regional realities, Earth’s climatic history, uncertainties in biogeochemical (BGC) and biogeophysical (BGP) changes following afforestation, and limitations in current assessment methodologies, emphasizing the need to reconcile these uncertainties before drawing firm conclusions about the climate benefits of afforestation. We hope that the identified gaps will drive the development of a more informed decision-making framework for Canadian afforestation policy, one that considers regional and future climatic contexts. Although we use the Canadian boreal as an example, most arguments in this review are applicable across the globe, particularly for the circumpolar nations.
Recommended citation: Dsouza, K. B., Ofosu, E., Salkeld, J., Boudreault, R., Moreno-Cruz, J., & Leonenko, Y. (2024). Assessing the climate benefits of afforestation: phenomenology, processes, methods. arXiv preprint arXiv:2407.14617. Full Document
talks
Representation learning for biology 1
Published:
The talks were about Epi-LSTM: A recurrent neural network model that builds cell type-agnostic representations of the human epigenome.
Representation learning for biology 2
Published:
The talks were about Hi-C-LSTM: a contact generation framework that forms Hi-C intrachromosomal representations.
teaching
ELEC 433: Error control coding for communications and computers
Course, The University of British Columbia, Department of ECE, 2018
The course included code design techniques, including Hamming, BCH, Reed-Solomon, LDPC and convolutional codes, ARQ techniques, and LFSR implementation of encoding-decoding algorithms. Created and graded assignments.
CPEN 491: ECE final year undergraduate Capstone design project
Projects, The University of British Columbia, Department of ECE, 2019
This course involved mentoring final year undergraduate capstone students. I provided design inputs at various stages and helped them to drive the projects to completion. For Data and ML related projects I provided substantial guidance each week (2019:1,2,3; 2020:1,2; 2021:1,2,3). All Coding was done by the students.
CPEN 481: Economic Analysis of Engineering Projects
Course, The University of British Columbia, Department of ECE, 2020
The course included Time-money relationships; economic analysis of alternatives including the effects of interest rates, inflation, depreciation, taxation and uncertainty; cost estimation and budgeting; financial analysis of engineering operations. Created and graded assignments.