Question: People are saying that if you're going to let immigrants in you must be willing to open your house to them.

This question does not correctly state the issue:
"They are saying that if we let them in 'you should open your house to house 3 or 4 of 'them.' "

The correct issue is different and could be stated as follows: "If you demand that the Caravan people be admitted despite my deep concerns for the potential financial costs to me as well as in terms of the safety and security of myself and my family, you should be willing to ensure that you take on your fair share of the possible costs and consequences by opening the gates of your gated community to immigration and directly sponsoring 3-4 members of the caravan."

You might respond, "That's silly. We all pay taxes, and will equally share the burden of supporting the new arrivals until they get established and become contributing members of our society." In the end, we will all be better off. "Besides that, it is the just, and fair, and right thing to do."

Let's look at the utilitarian argument more deeply via a cost/benefit analysis--in particular, Cui Bono, who benefits? In this situation, obviously the migrants would benefit. In addition, owners of large farms and factories would benefit from the availability of cheap labor. Wealthy people who live on islands, on mountain tops, and inside gated estates and protected communities would benefit from the cheap labor on their estates, and from enhanced financial portfolios. The average citizen would experience maximal levels of life satisfaction and benefit from being just, virtuous, and righteous and would undoubtedly be admitted directly to Heaven if there were such a place. But what about the costs? The taxpaying public would have to pay for the schools, healthcare, legal and welfare benefits for the new arrivals-- and even some additional law enforcement since we have all of these restrictive laws that a few of the migrants seem to be ignoring.
For the sake of argument, let's assume that the cost of supporting the migrants is $1000 per year for each taxpayer. Mr Smith earns 50 thousand dollars per year. Ms. Movie Star earns an average for 20 million dollars per year. That 1000 dollars means Mr. Smith's kids will have to forego dental care for the year. That 1000 dollars is what Ms. Movie Star usually tips her hairdresser at Christmas time.
Since there are many more Smith families that Ms. Movie Stars, I doubt the utilitarian argument would hold up, and we would not achieve the "best for the most" by admitting the Caravan. If this is correct, we then have to revert to arguments based on altruism, duty, justice, equality or fairness. But those are for another day.