Neal Wilson, Co-CEO of EJF Capital, says the labor market is "certainly slowing, but it's not slow" as the directional trend gets weaker..