Based on my research it seems like this can happen when the device supports both LE and BDR/EDR transports. I've created a test version of the BLE extension that forces LE connections starting with Android 6.0 Marshmallow and up (the API to do this was introduced in 6.0).
I've also added a DeviceSortMode property, which you can use to specify how the devices are sorted in the list. The default option is RSSI, which matches the old behavior. You may want to consider switching it to "Discovery" or "MAC" if you're finding that the devices are moving around too much for your taste. The version number is 20201223.
P.S. The sort is perfectly stable (Java uses mergesort as its sorting algorithm), it's just that the devices are sorted by signal strength which will change as the devices and objects move in space.