دانلود مقاله تعادل در مدل سازی شبکه های عصبی
چکیده
1. مقدمه
2. رگرسیون لجستیک
3. رگرسیون شبکه عصبی و توقف اولیه
4. تنظیم تعصب جهانی
5. مثال
6. نتیجه گیری
منابع
Abstract
1. Introduction
2. Logistic regression
3. Neural network regressions and early stopping
4. Global bias regularisation
5. Example
6. Conclusions
Notes on contributor
References
چکیده
در تئوری تخمین و پیش‌بینی، توجه قابل توجهی به مسئله وجود برآوردگرهای بی‌طرف در سطح جمعیت جهانی می‌شود. پیشرفت‌های اخیر در مدل‌سازی شبکه‌های عصبی عمدتاً بر دقت در سطح نمونه دانه‌ای متمرکز شده است و مسئله بی‌طرف‌بودن در سطح جمعیت تقریباً به طور کامل توسط آن جامعه نادیده گرفته شده است. ما این سوال را در مدل‌های رگرسیون شبکه عصبی مورد بحث قرار می‌دهیم و روش‌هایی برای دریافت برآوردگرهای بی‌طرف برای این مدل‌ها در سطح جمعیت جهانی ارائه می‌کنیم.
توجه! این متن ترجمه ماشینی بوده و توسط مترجمین ای ترجمه، ترجمه نشده است.
Abstract
In estimation and prediction theory, considerable attention is paid to the question of having unbiased estimators on a global population level. Recent developments in neural network modelling have mainly focused on accuracy on a granular sample level, and the question of unbiasedness on the population level has almost completely been neglected by that community. We discuss this question within neural network regression models, and we provide methods of receiving unbiased estimators for these models on the global population level.
Introduction
In recent years, neural networks have become state-of-the-art in all kinds of classification and regression problems. Snapshots of their history and their success are illustrated in LeCun et al. (2015) and Schmidhuber (2015). Their popularity is largely based on the facts that they offer much more modelling flexibility than classical statistical regression models (such as generalised linear models) and that increasing computational power combined with effective training methods have become available, see Rumelhart et al. (1986). Neural networks outperform many other classical statistical approaches in terms of predictive performance on an individual sample level, they allow to include unstructured data such as texts into the regression models, see Lee et al. (2020) for a word embedding example, and they allow for solving rather unconventional regression problems, see Cheng et al. (2020) and Gabrielli (2020) for examples. Therefore, our community has gradually been shifting from a data modelling culture to an algorithmic modelling culture, we refer the reader to Breiman (2001) and Shmueli(2010).
Conclusions
We have discussed the important problem of considering statistical models that provide unbiased mean estimates on a global population level (balance property). Classical statistical regression models like generalised linear model naturally have this balance property under the canonical link choice because the maximum likelihood estimator provides a critical value of the corresponding optimisation problem. In general, early stop gradient-descent calibrated neural networks fail to have the balance property, because early stopping prevents these models from taking parameters in critical points of the (deviance) loss function. In many applications, this does not reflect a favourable model calibration because it may lead to substantial price misspecification on a global population level. Therefore, we have proposed improvements that lead to globally unbiased solutions. These solutions include an additional generalised linear model optimisation step or shrinkage regularisation to empirical averages. The numerical example shows that we prefer the additional generalized linear model optimisation step.