Yeh, I am sure you could use that FFT code if you wanted to experiment with fast transform neural networks.
The FFT would act as a collection of non-adjustable dot products. Then you would use individually adjustable parametric activation functions.
Ie. f(x)=a.x x>=0, f(x)=b.x x<0. Where each individual activation function has its own adjustable parameters a and b.
Is it acceptable to change around the roles (being adjustable) of dot products and activation functions in such a way in a neural network? Actually it is. Especially if you understand ReLU as a switch.
On: f(x)=x (connect)
Off: f(x)=0 (disconnect)
If you think about an audio source switch on an audio amplifier. It may only have 2 states, on and off.
However when on it lets through a complicated audio signal.
Then a conventional ReLU neural network is just a switched composition of dot products.
The dot product of a number of dot products is still a dot product.
If you think about it, for a particular input to a ReLU network the state of each switch is definitely decided. Which dot products connect which other ones are all know and can be condensed down into a simpler equivalent dot product.
Then the output vector is a simple matrix mapping of the input vector.
Of course for each different input vector there is a different matrix, anyway pretty interesting.
