There are two lines of thought which lead to the understanding of fellowship as a necessary aspect of the Church.
First, Jesus has commanded that his disciples be taught to observe all that he previously commanded (Matt. 28:20). The two primary commands given by Jesus were to love God and to love our neighbor:
28 And one of the scribes came up and heard them disputing with one another, and seeing that he answered them well, asked him, “Which commandment is the most important of all?” 29 Jesus answered, “The most important is, ‘Hear, O Israel: The Lord our God, the Lord is one. 30 And you shall love the Lord your God with all your heart and with all your soul and with all your mind and with all your strength.’ 31 The second is this: ‘You shall love your neighbor as yourself.’ There is no other commandment greater than these.” 32 And the scribe said to him, “You are right, Teacher. You have truly said that he is one, and there is no other besides him. 33 And to love him with all the heart and with all the understanding and with all the strength, and to love one’s neighbor as oneself, is much more than all whole burnt offerings and sacrifices.” 34 And when Jesus saw that he answered wisely, he said to him, “You are not far from the kingdom of God.” And after that no one dared to ask him any more questions. Mark 12:28–34 (ESV)
When it comes to the matter of the church, Jesus was even more explicit as to the need of love among his disciples:
31 When he had gone out, Jesus said, “Now is the Son of Man glorified, and God is glorified in him. 32 If God is glorified in him, God will also glorify him in himself, and glorify him at once. 33 Little children, yet a little while I am with you. You will seek me, and just as I said to the Jews, so now I also say to you, ‘Where I am going you cannot come.’ 34 A new commandment I give to you, that you love one another: just as I have loved you, you also are to love one another. 35 By this all people will know that you are my disciples, if you have love for one another.” John 13:31–35 (ESV)
Indeed, love for one another must be the telling sign of actually being a disciple of Jesus. Thus, those who will not love are denying by their actions that they are a disciple.