First of all, I find it sus that grayImage is 8 bit, but the QRgb array is 32 bit. The memmove should either segfault or overwrite the data array with something random.

What format is the original QImage in? Is there a need to convert to grayscale? Yes, you should be able to just use setPixel on it.

Why are you so sure that using setPixel() is slower than manipulating the image through pointers? I doubt it hardly makes a difference.
If it's so time critical, let's see see some time measurements. Otherwise it would be much better to use the provided interface to manipulate the data.