update record：

• 2024.9 初建
• 2024.09.25 添加KA-Transformer、CKAN、KAN in low-data Regime等内容

【持续更新中····】

前言

KAN是MIT2024年5月提出，与传统MLP（多层感知机）并列的全新深度学习架构。截至2024.9.13已有150+次引用。笔者是该方向的入门新手，开此贴记录相关方向的论文调研笔记~（欢迎指正和补充）

KAN 1.0 原文链接

• 简洁的高效KAN开源代码 efficientKAN

KAN 2.0 原文链接

相关研究

basis function改进类

原始KAN使用B样条拟合每个node的一元非线性函数，该部分关注对基函数$\varphi (x)$进行调整/对比的研究。

《Kolmogorov-arnold networks are radial basis function networks》

• replaces the B-spline bases calculations with Gaussian RBF（径向基函数）

• 开源代码：Code

附：分别使用Fourier、Jacobi、Cheby替换基函数的开源代码

与其他DL场景结合类

DL领域衍生出了除常见深度MLP之外的其他学习框架，如针对隐私保护与分布式场景的联邦学习、针对时间维度考量的时序预测等。这一部分关注KAN与其他DL场景结合的相关研究

联邦学习

《F-KANs: Federated Kolmogorov-Arnold Networks》

• 每个client训练局部KAN，中心server集成模型
• 实验（代码link：Code）
  • 2 clients
  • dataset: Iris 数据集链接
  • summary: KAN model is not only accurate but also consistent in
    its predictions, capturing all true positives (300s vs 2s/ KAN100% accuracy)
  • The spline-based univariate features enable KAN to capture
    complex patterns quickly and accurately, resulting in stable
    and high performance with fewer training rounds.

时间序列

《KAN for Time Series Analysis》

• KANs outperforms conventional MLP in a real-world satellite traffic forecasting task, providing more accurate results with considerably fewer number of parameters.

• aim to evaluate the practicality of KANs in realworld scenarios, analyzing their efficiency in terms of the
  number of trainable parameters and discussing how the additional degrees of freedom might affect forecasting performance.

• using realworld satellite traffic data.

• 单变量时间序列预测

• 两层KAN，输入层和输出层的节点数分别对应total amount of time steps.

• 基于卫星数据预测交通状况

• 实验

  • 对比KAN与MLP架构的performance(6 beam area)
  • 用一周数据预测一天
  • KAN shows a rapid adjustment <> MLP exhibits a lag
  • KAN matched the rapid volume <> MLP moderately over/under-predicted
  • the robustness of KAN despite the complexity and higher volume <>
  • 参数方面：This reduced complexity suggests that KANs can achieve higher or comparable forecasting accuracy with simpler and potentially faster models.（用于资源有限和快速部署的场景）
    

《Kolmogorov-Arnold Networks (KANs) for Time Series Analysis》

• T-KAN:detect conecpt drift within time series(univariate)
  • core：use sliding window (two historical time steps to predict the next time step in the example, different KAN structures & activation functions represent different concepts) to observe the variations in KAN and identify concept drift.
    
• MT-KAN: imrove predictive performance (multivariate time series)
  

KAN4TSF: Are KAN and KAN-based models Effective for Time Series Forecasting?

• 代码：code

网络架构类

将KAN与其他基于MLP的架构“缝合”相关研究。

Tranformer

《Kolmogorov–Arnold Transformer》

卷积

《CKAN: Convolutional KAN models for intrusion Detection In IoT environment》

• replacing the Multi-Layer Perceptrons (MLPs) layers with Kolmogorov-Arnold Networks (KANs) layers inside the Convolutional Neural Networks (CNN) architecture.
• 在三个数据集（涉及二分类与多分类）上，CKAN效果优于other well-known Deep Learning (DL) models like CNN, recurrent neural networks (RNN), and Autoencoder.

其他

《KAN in low-Data Regimes： A comparative Study with MLP》

• Using empirical evaluations on simulated data and two real-world data sets from medicine and engineering, we explore the trade-offs between model complexity and accuracy, with particular attention to the role of network depth.
• Our findings show that MLPs with individualized activation functions achieve significantly higher predictive accuracy with only a modest increase in parameters, especially when the sample size is limited to around one hundred. For example, in a three-class classification problem within additive manufacturing, MLPs achieve a median accuracy of 0.91, significantly outperforming KANs, which only reach a median accuracy of 0.53 with default hyperparameters.