The major downside having both Fannie and you can Freddie: Its borrowing-score criteria are far more restrictive than FHA’s