Thurs Nedarim 74

The Gemara explains that R' Yehoshua that says that if there is only one Yavum, he may be Maifer the Yevama's oaths, it's because he holds there is Zika (that they're connected to the point that we consider him to be her husband.)

Ran asks: the Gemara there says we don't find anyone that holds Zika is so strong that we already consider him to be her husband except r' Oshia explaining R' Shimon, and he was disproved?

The Ran answers: the Gemara there only means that we don't find anyone that explains R' Shimon that Zika makes him her husband, however, we can say that R' Yehoshua held that way.

Alternatively, we don't consider him to be her complete husband, rather just being on the same plane as Kiddushin, that he can Maifer her Neder in partnership with her father's Hafara.