Soft Constraints¶
Soft constraints are a way of combining a priori theoretical information or opinions about the desired value of estimated parameters with what the data suggests the value of those parameters should be. The commands available for applying soft constraints are the con command, which applies a constraint directly to a single parameter or to a linear function of a number of parameters; and the sma command, which softly constrains a number of distributed lag weights to lie close to a certain degree of polynomial. The con command is explained below. See the distributed lags topic for details of the sma command.
- con <top> <c> = <linear function of ai’s>
This command imposes “softly” the given constraint on the regression. The <top> (trade-off paramater) determines how “soft” the constraint with the convention: “the higher, the harder.” Specifically, the constraint counts as \(top \times T\) observations, where T is the number of regular observations in the regression. Soft constraints also are known as “Theil’s mixed estimation,” as “stochastic constraints,” and also are a type of Bayesian regression.
Examples:
con 100 1 = a3 + a4 + a5 con 200 0 = a7 - 3a6 + 3a7 - a8
Ridge Regression
This technique amounts precisely to imposing a soft constraint that one or more of the coefficients should each be zero. For example,
con 100 0 = a2
con 100 0 = a3
will put <a2> and <a3> on a “ridge.” Such constraints may increase the t statistic for a variable, but they almost certainly distort the regression coefficients.
Quadratic Programming Regression
G7 also can estimate regression parameters subject to “hard” constraints. These may be combined with soft constraints.
Suppose matrix A specifies linear combinations of the regression parameters \(\beta\) that is less than or equal to vector b. Then we have the conditions that
Constraints A and b are imposed with the qpcon command, and the parameter vector \(\beta\) is estimated with the r command. Soft constraints also may be added with the con and sma commands. These constraints are imposed on the next linear regression, which is specified by the r command. If no constraints are imposed with the qpcon command, then G7 finds parameters using OLS. If constraints are imposed with qpcon, then G7 finds parameters using a quadratic programming algorithm.
The syntax for the qpcon command is as follows:
- qpcon b <sign> [constant][*]ai [<±> [constant][*] aj …]
where <sign> is <, =, or > (<= and >= also are accepted but are recorded as strict inequality constraints). <constant> are optional scalar multiples for the respective parameters. An asterisk may be included to clarify that the parameters are multiplied by the corresponding constant. The parameters are denoted by ai, where i is the position in the regression equation of the corresponding variable. If a constant is included in the regression, then the parameters are denoted as a1, a2, …. Right-hand-side terms are separated by either a + or -.
Examples:
qpcon 25.0 > a1 # constrain the constant qpcon 13.0 = 1*a2 + 1*a3 # constrain the sum of slope parameters
Additional details on soft constraints and quadratic programming are available in the paper presented at the 2004 Inforum World Conference by Ronald Horst.
