Logo: to the web site of Uppsala University

uu.sePublications from Uppsala University
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Random Features Model with General Convex Regularization: A Fine Grained Analysis with Precise Asymptotic Learning Curves
Chalmers Univ Technol, Gothenburg, Sweden..
Chalmers Univ Technol, Gothenburg, Sweden..
Uppsala University, Disciplinary Domain of Science and Technology, Technology, Department of Electrical Engineering, Signals and Systems.ORCID iD: 0000-0001-8978-2990
Chalmers Univ Technol, Gothenburg, Sweden..
2023 (English)In: Proceedings of The 26th International Conference on Artificial Intelligence and Statistics / [ed] Francisco Ruiz; Jennifer Dy; Jan-Willem van de Meent, JMIR Publications, 2023Conference paper, Published paper (Refereed)
Abstract [en]

We compute precise asymptotic expressions for the learning curves of least squares random feature (RF) models with either a separable strongly convex regularization or the ℓ1 regularization. We propose a novel multi-level application of the convex Gaussian min max theorem (CGMT) to overcome the traditional difficulty of finding computable expressions for random features models with correlated data. Our result takes the form of a computable 4-dimensional scalar optimization. In contrast to previous results, our approach does not require solving an often intractable proximal operator, which scales with the number of model parameters. Furthermore, we extend the universality results for the training and generalization errors for RF models to ℓ1 regularization. In particular, we demonstrate that under mild conditions, random feature models with elastic net or ℓ1 regularization are asymptotically equivalent to a surrogate Gaussian model with the same first and second moments. We numerically demonstrate the predictive capacity of our results, and show experimentally that the predicted test error is accurate even in the non-asymptotic regime.

Place, publisher, year, edition, pages
JMIR Publications, 2023.
Series
Proceedings of Machine Learning Research, ISSN 2640-3498 ; 206
National Category
Probability Theory and Statistics
Identifiers
URN: urn:nbn:se:uu:diva-549028ISI: 001298469305022OAI: oai:DiVA.org:uu-549028DiVA, id: diva2:1933462
Conference
26th International Conference on Artificial Intelligence and Statistics (AISTATS), April 25-27, 2023, Valencia, Spain
Available from: 2025-01-31 Created: 2025-01-31 Last updated: 2025-11-17Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Paper in full-text

Authority records

Özcelikkale, Ayca

Search in DiVA

By author/editor
Özcelikkale, Ayca
By organisation
Signals and Systems
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 55 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf