For I bear them witness that they have a zeal for God, but not according to knowledge. (ESV)
Many times I have heard people declare that as long as one is sincere that is all that matters. The Apostle Paul is recognizing here that the Jews were not lacking in sincerity. They were zealous for God. However, he is also saying that this zeal is not sufficient for salvation. They had zeal for God, but not according to knowledge.
Paul loved his Jewish brothers and sisters with his whole heart. He recognized their zeal, or sincerity. He also loved them enough to point out to them that their sincerity was not going to be enough to keep them from the coming wrath of God against their sin. The sacrifice of Jesus Christ was the way of salvation, not their sincere efforts to keep the Law.
What is interesting is that the zeal of the Jews led them to the persecution of the Church. When Paul describes his old life in Philippians he says, "as to zeal, a persecutor of the church; as to righteousness under the law, blameless." The very means of salvation was through the church. The Jews were persecuting the very means God had provided for their salvation.
Consider that Paul himself was on the road to Damascus. He was going with orders from the Jews in Jerusalem to throw Christians into prison. As he was on the way there God appeared to him and he was thrown to the ground and saw a blinding light. What did God say to him? "I am Jesus, whom you are persecuting." The zeal of the Jews led them to persecute the church. They were persecuting Jesus.
Being sincere is not enough. If your sincerity does not lead you to the knowledge and worship of Jesus Christ, it is a zeal without knowledge.