Tolstoy, these are some great questions.
First, you have to understand that even among orthodox jews, there are quite a few groups. Not everything I will say here can be applied to all orthodox jews.
Regarding taxes:
These people rarely work. The men study all day in "Yeshivot" (very religious schools that teach only religious writings - I even can't call it a school). By men, I mean men of all ages - it starts with a very early age and can go on for the rest of their lives. They do receive some sort of social insurance. Since they have a lot of children (5 children is not considered much by their standards), they receive a enough social insurance for the living. Their women work in a lot of jobs and get paid with cash, and they work in very small businesses that operate in the community - I am not even sure they are registered as businesses. As such, it is very easy not to pay taxes, and in fact not paying tax is considered as something positive. There are no 'ultra-orthodox companies' as such, but only small businesses that operate to provide the immediate needs of this society.
New immigrants indeed get tax reduction, but for a limited amount of time and money.
Regarding education system:
The orthodox schools are funded by the government, but the study is mostly (95%+) religious. They only get to study the most basic math and the most basic English. The government tries to enforce so called "core-study" professions, such as math, literature, English, citizenship, but these schools rarely follow this program. This is one of the most hot-debated topics in Israel - whether the government should or should not support such schools.
It is definitely not home schooling.
Regarding courts:
The ultra-orthodox groups do not recognize the court, which is mostly secular. For any disputes they go to the community rabi. When it comes to crimes, the community often defends the offenders, hides them, and makes the task of finding the offenders extremely difficult for the police.
The reason these groups can get away with all these things is that... well, you wouldn't believe it, but in Israel there is a great amount of tolerance towards minorities. The problem is that such groups become very big, to the point they have influence on the democracy as whole. Religious parties like Shas protect these people and deliver a lot of funds to them. In fact, Shas is almost always in the coalition - were there very few governments that Shas was not a part of. In addition, you cannot target a specific group, so if you want to pay less social security fees per child, other religious and more law-abiding groups with a lot of children will protest, and you can't really starve the children. These groups see giving birth to as many children as possible as a "Mizvah" - something God told to do.
Not all religious people behave like this. There are dozens of groups, some big and some small, some even are not considered groups, that religious jews divided by. There are "religious-light" Jews, which behave almost as secular people, with the biggest difference is that they don't work at Shabbath and don't light fire/light on Shabbath. There are more religious groups which send their children to schools divided by gender - school for boys and school for girls. I can go on and on...
Unlike what people tend to think, the Israeli society is very diverse. I did not even start to talk about Arabs, Beduins and Druze and the difference between them (for example, most Arabs don't serve in the army due to: 1. it is inhumane to force an Arab to confront someone who might be his family and 2. there is always the issue of loyalty to the state of Israel; while most Druze and some Beduins do serve in the army). There is the so-called "Beduin problem" - the Beduins, being mainly nomadic, have lands in southern Israel (the Negev desert) and their rights to some of the lands are not recognized by Israel since these rights originate fromthe Ottoman period and most of them don't have papers that confirm they own this land. Furthermore, the more modern Beduins, mostly people of age 40-, are more accustomed to living inside cities and they are not connected to the land as their fathers are.
Heh, you can write a lot of books about the Israeli population...
If you have more questions, please feel free to ask.