Abstract:

As a multi-task meta learning algorithm， Model Agnostic Meta Learning （MAML） can use different models and adapt quickly to different tasks， but it still needs to be improved in terms of training speed and accuracy. The principle of MAML was analyzed from the perspective of Gaussian stochastic process， and a new Model Agnostic Meta Learning algorithm based on Bayesian Weight function （BW-MAML） was proposed， in which the weight was assigned by Bayesian analysis. In the training process of BW-MAML， each sampling task was regarded as following a Gaussian distribution， and the importance of the task was determined according to the probability of the task in the distribution， and then the weight was assigned according to the importance， thus improving the utilization of information in each gradient descent. The small sample image learning experimental results on Omniglot and Mini-ImageNet datasets show that by adding Bayesian weight function， for training effect of BW-MAML after 2500 step with 6 tasks， the accuracy of BW-MAML is at most 1.9 percentage points higher than that of MAML， and the final accuracy is 0.907 percentage points higher than that of MAML on Mini-ImageNet averagely； the accuracy of BW-MAML on Omniglot is also improved by up to 0.199 percentage points averagely.

Key words: Bayesian analysis, Gaussian stochastic process, machine learning, meta learning, few-shot learning

摘要：

模型无关的元学习（MAML）是一种多任务的元学习算法，能使用不同的模型，并快速地在不同任务之间进行适应，但MAML在训练速度与准确率上还亟待提高。从高斯随机过程的角度出发对MAML的原理进行分析，提出一种基于贝叶斯权函数的模型无关元学习（BW-MAML）算法，该权函数利用贝叶斯分析设计并用于损失的加权。训练过程中，BW-MAML将每次抽样的任务视为遵循高斯分布，根据贝叶斯分析计算不同任务在分布中的概率，并根据任务在分布中的概率判断该任务重要程度，再以此赋以不同的权重，从而提高每次梯度下降中信息的利用率。在Omniglot与Mini-ImageNet数据集上的小样本图像学习实验结果表明，通过增加贝叶斯权函数，BW-MAML的训练效果在6任务训练2 500步后，在Mini-ImageNet上的准确率比MAML的准确率最高提高了1.9个百分点，并且最终准确率比MAML平均提升了0.907个百分点；在Omniglot上的准确率也平均提升了0.199个百分点。

关键词: 贝叶斯分析, 高斯随机过程, 机器学习, 元学习, 小样本学习