You're probably correct about that. I think part of the refugium is for the fun of having the little critters in there, and to be able to check if it increases stability by lighting it overnight.
You gave me a good idea though, I can instead have the refugium as an insert box. Then if I get sick of changing filter socks, or if the refugium is too small to be any good, then I can later just remove the fuge and swap over to a roller instead.