A Kalman filter is a way to take noisy measurments (accelerations, gyro rate of change) and create estimate of what the actual "state" of your system is (attitude, speed).
Math is a fact of life. Newton (and others) didn't invent calculus just to make life hard for students - they invented it because that is the tool that is needed to describe how the physical world operates.
You don't need to use a kalman filter, but then you would need to design an algorithm to translate from the sensor inputs to the actual state of your quadcopter. If you know the magnitude of the noise the Kalman filter will give you the "optimal" estimate of those states. For other methods you will have to sort out the gains of your estimator on your own.