• 58 Posts
  • 105 Comments
Joined 1 year ago
cake
Cake day: June 16th, 2023

help-circle



















  • Neuronal firing is often understood as a fundamentally binary process, because a neuron either fires an action potential or it does not. This is often referred to as the “all-or-none” principle.

    Isn’t this true of standard multi-bit neural networks too? This seems to be what a nonlinear activation function achieves: translating the input values into an all-or-nothing activation.

    The characteristic of a 1-bit model is not that its activations are recorded in a single but but that its weights are. There are no gradations of connection weights: they are just on or off. As far as I know, that’s different from both standard neural nets and from how the brain works.






  • The academics I know are all pretty miserable these days. They can see that it’s a corrupt, exploitative system and they feel powerless to change it. They spend their time writing grant applications and chasing money, then pumping out papers they know are fairly trivial, but they have to write them to keep the funding coming in. Some of the scientific disciplines are in a slow state of crisis due to a serious loss of confidence in the credibility or value of much of the research. And the younger ones know they’ll never get tenure and are on a shit career track potentially forever. But even the ones with tenure seem pretty unhappy, working for these organizations that relentlessly seek money and superficial prestige.

    This is so far from what academia ought to be about, and from the enthusiasms that brought these people into it in the first place. I got out 20 years ago because I found this stuff repellent then. It’s worse now. it’s sad that our society can’t provide a place for smart and enthusiastic people to do honest research without all this corrupting quasi-commercial (or sometimes simply commercial) influence.