A neural network with "clearning" and continuity constraint is described. When this neural network is trained, not only the weights, but also the input to the network are adjusted, to minimize a cost function consisting of three terms: The first term measures the difference between the network output and the data (the output constraint), the second term measures the difference between the network input and the data (the input constraint), and the third term measures the difference between the network output and the network input of the next step (the continuity constraint). A preliminary test on the Mackey-Glass time series shows that the new network gives better performance than a traditional neural network when there is noise in the time series. Application to seasonal climate forecasting is also presented.