Statistical Properties of Numerical Derivatives and Algorithms

数值导数和算法的统计性质

• 批准号: 1025035
• 负责人: Denis Nekipelov
• 金额: $ 13.72万
• 依托单位: University of California-Berkeley
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-09-15 至 2013-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1025035&HistoricalAwards=false
• 关键词: Statistical; Properties; Numerical; Derivatives; Algorithms

Numerical differentiation is widely used in econometrics and many other areas of quantitative economic analysis. Many functions that need to be differentiated in econometric analysis need to be estimated from the data. For example, estimating the approximate variance of an estimator often requires estimating the derivatives of the moment conditions that define the estimator. Many estimators are also obtained by finding the zeros of the first order condition of the sample objective functions.The estimated functions can be either non-differentiable or difficult to differentiate analytically. Oftentimes the estimated functions are complex and can be challenging to compute even numerically. Empirical researchers often apply numerical differentiation methods which depend on taking a finite number of differences of the objective function at discrete points, either explicitly or implicitly through the use of software routines, to the estimated functions from the sample in order to approximate the derivative of the unknown true functions.A key tuning parameter that determines how well the numerical derivatives approximate the analytic derivatives is the step size used in the finite differencing operation. Empirical researchers often find that different step sizes can lead to very different numerical derivative estimates. While the importance of numerical derivatives has not gone unnoticed in econometrics, statistics and mathematics, the results that are available in the existing literature are very limited in scope.The goal of this project is to take an important step to provide a systematic framework for understanding the conditions on the step size in numerical differentiation that are needed to obtain the optimal quality of approximation. These conditions involve subtle tradeoffs between the complexity of the function that needs to be differentiated and the amount of information that is available in the sample of data, and the degree of smoothness of the expectation of the function with respect to the sampling distribution. Empirical process theory provides a powerful tool for analyzing the complex of functions in the presence of randomly sampled data.This project focuses on analyzing the use of numerical derivatives in estimating the asymptotic variance of estimators and in obtaining extreme estimators through gradient based optimization routines. The PIs' first goal is to give general sufficient consistency conditions that allow for nondifferentiable and discontinuous moment functions in consistent variance estimation. The precise rate conditions for the step size in numerical differentiation that we obtain depend on the tradeoff between bias and the degree of nonsmoothness of the moment condition. These general conditions can be specialized for certain continuous models, for which choosing a smaller step size can only be beneficial in reducing the asymptotic bias. However, the asymptotic bias will be dominated by the statistical noise once it falls below a certain threshold. The second goal of this project is to analyze a class of estimators that are based on numerically differentiating a finite sample objective function, and provide conditions under which numerical derivative based optimization methods deliver consistent and asymptotic normal parameter estimates. The conditions for numerical extreme estimators require that the step size used in the numerical derivative converge to zero at specific rates when the sample size increases to infinity. The conditions required for the consistency of the asymptotic variance and for the convergence of the estimator itself can be different. The PIs seek extensive results that cover finite dimensional parametric models, infinite dimensional semiparametric models, and models that are defined by U-processes involving multiple layers of summation over the sampling data. The proposed project involves joint work with Professor Aprajit Mahajan from Stanford University.

数值微分广泛应用于计量经济学和定量经济分析的许多其他领域。计量经济学分析中需要微分的许多函数都需要根据数据进行估计。例如，估计估计器的近似方差通常需要估计定义估计器的矩条件的导数。许多估计量也是通过找到样本目标函数的一阶条件的零点来获得的。估计函数可以是不可微的，也可以是难以解析微分的。 通常，估计的函数很复杂，即使是数值计算也具有挑战性。经验研究人员经常应用数值微分方法，该方法依赖于通过使用软件例程显式或隐式地在离散点处对目标函数进行有限数量的差异，以估计样本中的函数，以近似估计函数的导数未知的真实函数。决定数值导数与解析导数的近似程度的关键调整参数是有限差分运算中使用的步长。经验研究人员经常发现不同的步长会导致非常不同的数值导数估计。虽然数值导数的重要性在计量经济学、统计学和数学中并没有被忽视，但现有文献中可用的结果范围非常有限。该项目的目标是迈出重要一步，为理解提供系统框架数值微分中获得最佳近似质量所需的步长条件。这些条件涉及需要微分的函数的复杂性和数据样本中可用的信息量以及函数相对于采样分布的期望的平滑程度之间的微妙权衡。经验过程理论提供了一个强大的工具，可以在存在随机采样数据的情况下分析复杂的函数。该项目重点分析数值导数在估计估计量的渐近方差中的使用以及通过基于梯度的优化例程获得极端估计量。 PI 的首要目标是给出一般的充分一致性条件，允许在一致方差估计中使用不可微分和不连续的矩函数。我们获得的数值微分步长的精确速率条件取决于偏差和矩条件的非平滑程度之间的权衡。这些一般条件可以专门用于某些连续模型，对于这些连续模型，选择较小的步长只会有利于减少渐近偏差。然而，一旦渐近偏差低于某个阈值，就会受到统计噪声的支配。该项目的第二个目标是分析一类基于对有限样本目标函数进行数值微分的估计器，并提供基于数值导数的优化方法提供一致且渐近的正态参数估计的条件。数值极值估计量的条件要求当样本量增加到无穷大时，数值导数中使用的步长以特定速率收敛到零。渐近方差的一致性和估计器本身的收敛所需的条件可以不同。 PI 寻求广泛的结果，涵盖有限维参数模型、无限维半参数模型以及由涉及采样数据多层求和的 U 过程定义的模型。拟议的项目涉及与斯坦福大学 Aprajit Mahajan 教授的合作。