I'll post this here for posterity. It IS possible
int width = 960, height = 540;
SoftwareBitmap soft = new SoftwareBitmap(BitmapPixelFormat.Bgra8, width, height);
...
var buffer = new Windows.Storage.Streams.Buffer ( (uint)( 4 * width * height));
soft.CopyToBuffer(buffer);
DataReader dr = DataReader.FromBuffer(buffer);
var bytes = new byte[4 * width * height];
dr.ReadBytes(bytes);