journal
Journals Advances in Neural Information...

Advances in Neural Information Processing Systems

https://read.qxmd.com/read/38505104/okridge-scalable-optimal-k-sparse-ridge-regression
#1
JOURNAL ARTICLE
Jiachang Liu, Sam Rosen, Chudi Zhong, Cynthia Rudin
We consider an important problem in scientific discovery, namely identifying sparse governing equations for nonlinear dynamical systems. This involves solving sparse ridge regression problems to provable optimality in order to determine which terms drive the underlying dynamics. We propose a fast algorithm, OKRidge, for sparse ridge regression, using a novel lower bound calculation involving, first, a saddle point formulation, and from there, either solving (i) a linear system or (ii) using an ADMM-based approach, where the proximal operators can be efficiently evaluated by solving another linear system and an isotonic regression problem...
December 2023: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/38434255/large-language-models-transition-from-integrating-across-position-yoked-exponential-windows-to-structure-yoked-power-law-windows
#2
JOURNAL ARTICLE
David Skrill, Sam V Norman-Haignere
Modern language models excel at integrating across long temporal scales needed to encode linguistic meaning and show non-trivial similarities to biological neural systems. Prior work suggests that human brain responses to language exhibit hierarchically organized "integration windows" that substantially constrain the overall influence of an input token (e.g., a word) on the neural response. However, little prior work has attempted to use integration windows to characterize computations in large language models (LLMs)...
December 2023: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/38288081/learning-dynamics-of-deep-linear-networks-with-multiple-pathways
#3
JOURNAL ARTICLE
Jianghong Shi, Eric Shea-Brown, Michael A Buice
Not only have deep networks become standard in machine learning, they are increasingly of interest in neuroscience as models of cortical computation that capture relationships between structural and functional properties. In addition they are a useful target of theoretical research into the properties of network computation. Deep networks typically have a serial or approximately serial organization across layers, and this is often mirrored in models that purport to represent computation in mammalian brains...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/38149289/learning-concept-credible-models-for-mitigating-shortcuts
#4
JOURNAL ARTICLE
Jiaxuan Wang, Sarah Jabbour, Maggie Makar, Michael Sjoding, Jenna Wiens
During training, models can exploit spurious correlations as shortcuts, resulting in poor generalization performance when shortcuts do not persist. In this work, assuming access to a representation based on domain knowledge ( i.e., known concepts ) that is invariant to shortcuts, we aim to learn robust and accurate models from biased training data. In contrast to previous work, we do not rely solely on known concepts, but allow the model to also learn unknown concepts. We propose two approaches for mitigating shortcuts that incorporate domain knowledge, while accounting for potentially important yet unknown concepts...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/38031583/bivariate-causal-discovery-for-categorical-data-via-classification-with-optimal-label-permutation
#5
JOURNAL ARTICLE
Yang Ni
Causal discovery for quantitative data has been extensively studied but less is known for categorical data. We propose a novel causal model for categorical data based on a new classification model, termed classification with optimal label permutation (COLP). By design, COLP is a parsimonious classifier, which gives rise to a provably identifiable causal model. A simple learning algorithm via comparing likelihood functions of causal and anti-causal models suffices to learn the causal direction. Through experiments with synthetic and real data, we demonstrate the favorable performance of the proposed COLP-based causal model compared to state-of-the-art methods...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37766938/strokerehab-a-benchmark-dataset-for-sub-second-action-identification
#6
JOURNAL ARTICLE
Aakash Kaku, Kangning Liu, Avinash Parnandi, Haresh Rengaraj Rajamohan, Kannan Venkataramanan, Anita Venkatesan, Audre Wirtanen, Natasha Pandit, Heidi Schambra, Carlos Fernandez-Granda
Automatic action identification from video and kinematic data is an important machine learning problem with applications ranging from robotics to smart health. Most existing works focus on identifying coarse actions such as running, climbing, or cutting vegetables, which have relatively long durations and a complex series of motions. This is an important limitation for applications that require identification of more elemental motions at high temporal resolution. For example, in the rehabilitation of arm impairment after stroke, quantifying the training dose (number of repetitions) requires differentiating motions with sub-second durations...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37719469/hybrid-neural-autoencoders-for-stimulus-encoding-in-visual-and-other-sensory-neuroprostheses
#7
JOURNAL ARTICLE
Jacob Granley, Lucas Relic, Michael Beyeler
Sensory neuroprostheses are emerging as a promising technology to restore lost sensory function or augment human capabilities. However, sensations elicited by current devices often appear artificial and distorted. Although current models can predict the neural or perceptual response to an electrical stimulus, an optimal stimulation strategy solves the inverse problem: what is the required stimulus to produce a desired response? Here, we frame this as an end-to-end optimization problem, where a deep neural network stimulus encoder is trained to invert a known and fixed forward model that approximates the underlying biological system...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37614415/local-spatiotemporal-representation-learning-for-longitudinally-consistent-neuroimage-analysis
#8
JOURNAL ARTICLE
Mengwei Ren, Neel Dey, Martin A Styner, Kelly N Botteron, Guido Gerig
Recent self-supervised advances in medical computer vision exploit the global and local anatomical self-similarity for pretraining prior to downstream tasks such as segmentation. However, current methods assume i.i.d. image acquisition, which is invalid in clinical study designs where follow-up longitudinal scans track subject-specific temporal changes. Further, existing self-supervised methods for medically-relevant image-to-image architectures exploit only spatial or temporal self-similarity and do so via a loss applied only at a single image-scale, with naive multi-scale spatiotemporal extensions collapsing to degenerate solutions...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37534101/a-benchmark-for-compositional-visual-reasoning
#9
JOURNAL ARTICLE
Aimen Zerroug, Mohit Vaishnav, Julien Colin, Sebastian Musslick, Thomas Serre
A fundamental component of human vision is our ability to parse complex visual scenes and judge the relations between their constituent objects. AI benchmarks for visual reasoning have driven rapid progress in recent years with state-of-the-art systems now reaching human accuracy on some of these benchmarks. Yet, there remains a major gap between humans and AI systems in terms of the sample efficiency with which they learn new visual reasoning tasks. Humans' remarkable efficiency at learning has been at least partially attributed to their ability to harness compositionality - allowing them to efficiently take advantage of previously gained knowledge when learning new tasks...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37533756/class-aware-adversarial-transformers-for-medical-image-segmentation
#10
JOURNAL ARTICLE
Chenyu You, Ruihan Zhao, Fenglin Liu, Siyuan Dong, Sandeep Chinchali, Ufuk Topcu, Lawrence Staib, James S Duncan
Transformers have made remarkable progress towards modeling long-range dependencies within the medical image analysis domain. However, current transformer-based models suffer from several disadvantages: (1) existing methods fail to capture the important features of the images due to the naive tokenization scheme; (2) the models suffer from information loss because they only consider single-scale feature representations; and (3) the segmentation label maps generated by the models are not accurate enough without considering rich semantic contexts and anatomical textures...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37529401/amortized-inference-for-heterogeneous-reconstruction-in-cryo-em
#11
JOURNAL ARTICLE
Axel Levy, Gordon Wetzstein, Julien Martel, Frédéric Poitevin, Ellen D Zhong
Cryo-electron microscopy (cryo-EM) is an imaging modality that provides unique insights into the dynamics of proteins and other building blocks of life. The algorithmic challenge of jointly estimating the poses, 3D structure, and conformational heterogeneity of a biomolecule from millions of noisy and randomly oriented 2D projections in a computationally efficient manner, however, remains unsolved. Our method, cryoFIRE, performs ab initio heterogeneous reconstruction with unknown poses in an amortized framework, thereby avoiding the computationally expensive step of pose search while enabling the analysis of conformational heterogeneity...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37465369/harmonizing-the-object-recognition-strategies-of-deep-neural-networks-with-humans
#12
JOURNAL ARTICLE
Thomas Fel, Ivan Felipe, Drew Linsley, Thomas Serre
The many successes of deep neural networks (DNNs) over the past decade have largely been driven by computational scale rather than insights from biological intelligence. Here, we explore if these trends have also carried concomitant improvements in explaining the visual strategies humans rely on for object recognition. We do this by comparing two related but distinct properties of visual strategies in humans and DNNs: where they believe important visual features are in images and how they use those features to categorize objects...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37397786/manifold-interpolating-optimal-transport-flows-for-trajectory-inference
#13
JOURNAL ARTICLE
Guillaume Huguet, D S Magruder, Alexander Tong, Oluwadamilola Fasina, Manik Kuchroo, Guy Wolf, Smita Krishnaswamy
We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) that learns stochastic, continuous population dynamics from static snapshot samples taken at sporadic timepoints. MIOFlow combines dynamic models, manifold learning, and optimal transport by training neural ordinary differential equations (Neural ODE) to interpolate between static population snapshots as penalized by optimal transport with manifold ground distance. Further, we ensure that the flow follows the geometry by operating in the latent space of an autoencoder that we call a geodesic autoencoder (GAE)...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37362058/learning-low-dimensional-generalizable-natural-features-from-retina-using-a-u-net
#14
JOURNAL ARTICLE
Siwei Wang, Benjamin Hoshal, Elizabeth A de Laittre, Olivier Marre, Michael J Berry, Stephanie E Palmer
Much of sensory neuroscience focuses on presenting stimuli that are chosen by the experimenter because they are parametric and easy to sample and are thought to be behaviorally relevant to the organism. However, it is not generally known what these relevant features are in complex, natural scenes. This work focuses on using the retinal encoding of natural movies to determine the presumably behaviorally-relevant features that the brain represents. It is prohibitive to parameterize a natural movie and its respective retinal encoding fully...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37332888/information-bottleneck-theory-of-high-dimensional-regression-relevancy-efficiency-and-optimality
#15
JOURNAL ARTICLE
Vudtiwat Ngampruetikorn, David J Schwab
Avoiding overfitting is a central challenge in machine learning, yet many large neural networks readily achieve zero training loss. This puzzling contradiction necessitates new approaches to the study of overfitting. Here we quantify overfitting via residual information, defined as the bits in fitted models that encode noise in training data. Information efficient learning algorithms minimize residual information while maximizing the relevant bits, which are predictive of the unknown generative models. We solve this optimization to obtain the information content of optimal algorithms for a linear regression problem and compare it to that of randomized ridge regression...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37309509/seeing-the-forest-and-the-tree-building-representations-of-both-individual-and-collective-dynamics-with-transformers
#16
JOURNAL ARTICLE
Ran Liu, Mehdi Azabou, Max Dabagia, Jingyun Xiao, Eva L Dyer
Complex time-varying systems are often studied by abstracting away from the dynamics of individual components to build a model of the population-level dynamics from the start. However, when building a population-level description, it can be easy to lose sight of each individual and how they contribute to the larger picture. In this paper, we present a novel transformer architecture for learning from time-varying data that builds descriptions of both the individual as well as the collective population dynamics...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37192934/augmentations-in-hypergraph-contrastive-learning-fabricated-and-generative
#17
JOURNAL ARTICLE
Tianxin Wei, Yuning You, Tianlong Chen, Yang Shen, Jingrui He, Zhangyang Wang
This paper targets at improving the generalizability of hypergraph neural networks in the low-label regime, through applying the contrastive learning approach from images/graphs (we refer to it as HyperGCL ). We focus on the following question: How to construct contrastive views for hypergraphs via augmentations? We provide the solutions in two folds. First, guided by domain knowledge, we fabricate two schemes to augment hyperedges with higher-order relations encoded, and adopt three vertex augmentation strategies from graph-structured data...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37168261/efficient-coding-channel-capacity-and-the-emergence-of-retinal-mosaics
#18
JOURNAL ARTICLE
Na Young Jun, Greg D Field, John M Pearson
Among the most striking features of retinal organization is the grouping of its output neurons, the retinal ganglion cells (RGCs), into a diversity of functional types. Each of these types exhibits a mosaic-like organization of receptive fields (RFs) that tiles the retina and visual space. Previous work has shown that many features of RGC organization, including the existence of ON and OFF cell types, the structure of spatial RFs, and their relative arrangement, can be predicted on the basis of efficient coding theory...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37151570/outsourcing-training-without-uploading-data-via-efficient-collaborative-open-source-sampling
#19
JOURNAL ARTICLE
Junyuan Hong, Lingjuan Lyu, Jiayu Zhou, Michael Spranger
As deep learning blooms with growing demand for computation and data resources, outsourcing model training to a powerful cloud server becomes an attractive alternative to training at a low-power and cost-effective end device. Traditional outsourcing requires uploading device data to the cloud server, which can be infeasible in many real-world applications due to the often sensitive nature of the collected data and the limited communication bandwidth. To tackle these challenges, we propose to leverage widely available open-source data , which is a massive dataset collected from public and heterogeneous sources (e...
December 2022: Advances in Neural Information Processing Systems
https://read.qxmd.com/read/37101843/distinguishing-learning-rules-with-brain-machine-interfaces
#20
JOURNAL ARTICLE
Jacob P Portes, Christian Schmid, James M Murray
Despite extensive theoretical work on biologically plausible learning rules, clear evidence about whether and how such rules are implemented in the brain has been difficult to obtain. We consider biologically plausible supervised- and reinforcement-learning rules and ask whether changes in network activity during learning can be used to determine which learning rule is being used. Supervised learning requires a credit-assignment model estimating the mapping from neural activity to behavior, and, in a biological organism, this model will inevitably be an imperfect approximation of the ideal mapping, leading to a bias in the direction of the weight updates relative to the true gradient...
December 2022: Advances in Neural Information Processing Systems
journal
journal
32531
1
2
Fetch more papers »
Fetching more papers... Fetching...
Remove bar
Read by QxMD icon Read
×

Save your favorite articles in one place with a free QxMD account.

×

Search Tips

Use Boolean operators: AND/OR

diabetic AND foot
diabetes OR diabetic

Exclude a word using the 'minus' sign

Virchow -triad

Use Parentheses

water AND (cup OR glass)

Add an asterisk (*) at end of a word to include word stems

Neuro* will search for Neurology, Neuroscientist, Neurological, and so on

Use quotes to search for an exact phrase

"primary prevention of cancer"
(heart or cardiac or cardio*) AND arrest -"American Heart Association"

We want to hear from doctors like you!

Take a second to answer a survey question.