Weird order of switch-to-buffer with packages that augment completing-read?

I'm using vertico, consult, consult embark, and orderless and it's OK but for switch-to-buffer specifically it kind of bugs me that the order isn't connected to the last-used-order (the order that list-all-buffers uses). I'm like "I just used that buffer three seconds ago and now I need to search for it?!"

Help please? 🙏🏻 ♥ @[email protected]