The Bible indicates that it is possible to claim to be a Christian and yet not be one. For instance, in the parable of the wheat and tares, the tares look like wheat until fruition; and in the parable of the sower, some seed falls on rocky and thorny soil, both sprout (look like good fruit will grow), but eventually die.
Puritan Thomas Manton discusses an upright Christian walk in a sermon on Psalm 19:13, which you can read in full here.
Secondly, Positively. He that is upright may be known and tried by these four things – (1.) His principles; (2.) His conversation [behavior]; (3.) His ends; (4.) The fruits and effects of his constant endeavours.
1. His principles, which are –
[1.] The new nature fitting and suiting his heart to the things of God, so that
he loveth them not only out of interest, but inclination: Ps. cxix. 140, ‘Thy word is very pure, therefore thy servant loveth it.’ Surely he is upright that is thus naturally carried to the things of God. In some measure the man is restored to that frame of heart which mankind had in innocency; his heart inclined him to God. God made man upright; grace giveth somewhat of this: Heh. viii. 10, ‘I will put my laws into their minds, and write them in their hearts.’
[2.] By his love to Christ: 2 Cor. v. 14, ‘The love of Christ constraineth us.’ That cureth his self-love, which maketh us act crookedly and perversely. Self is his principle, self his business, self his end; his own contentment and satisfaction is all that he looketh at. But the love of Christ maketh him readily go about those things which are pleasing to Christ, and hath a mighty force and efficacy upon the soul to overrule our self-love, that we may not live to ourselves, but unto God, and obey his will.
2. By his conversation [behavior], wherein he betrayeth [shows] a constant care –
[1.] To avoid all known sin; to weaken the lusts, to suppress the acts: 1 Peter ii. 11, ‘Abstain from fleshly lusts, which war against the soul.’
But especially he is most careful to avoid his own personal sin: Ps. xviii. 23, ‘I was upright before thee, and kept myself from mine iniquity;’ which the affections of his own heart might most transport him into. He can sacrifice his Isaac, cut off his right hand, pluck out his right eye,
laboureth to subdue his particular corrupt inclinations, useth no guile to cloak and extenuate them before God: Ps. xxxii. 2, ‘Blessed is the man unto whom the Lord imputeth not iniquity, and
in whose spirit there is no guile.’
This is the man who may comfort himself with God’s acceptance, and the comforts and privileges of the new covenant.
[2.] As to the other part of his conversation, his way and the constant tenor of his walk is to please God, and he maketh conscience of obeying the will of God in all his actions: Phil. i. 10, 11, ‘That ye may approve things that are excellent, and ye may be sincere and without offence’ till the day of Christ, being filled with the fruits of righteousness, which are by Jesus Christ unto the praise and glory of God.’ His aim is at the constant practice of every thing that is good: ‘And having learned how to walk and to please God, so you would abound therein more and more,’ 1 Thes. iv. 1.
3. By his ends, which are the pleasing and glorifying of God. This is his main fixed scope: 2 Cor. v. 9, ‘Wherefore we labour, that, whether present or absent, we may be accepted of him;’ 1 Cor. x. 31, ‘Whether ye eat or drink, or whatever ye do, do all to the glory of God;’ Col. i. 10, ‘That ye might walk worthy of the Lord unto all pleasing, being fruitful in every good work, and increasing in the knowledge of God.’ There is no corrupt design of vainglory, credit, or by-respect, but [so] what he doth for God it is to God [and not for these previously mentioned reasons, because those corrupt designs are not there].
4. The effects and fruits, which is not an exemption from sin altogether, but a growth of the contrary principle, and the flesh is brought every day more and more into a subjection to the Spirit, and Christ’s interest prevaileth in the soul. The Spirit gets above the flesh, and the prevailing bent of the heart is set and fixed towards God.
Either sin reigneth or grace must reign; something must be in solio, in the throne. That which is in the throne showeth the state of the heart, be it sin or grace, and that which generally and mostly commandeth and influenceth our conversations, that is in the throne. As the sun in the midst of heaven sendeth abroad his influences on every side, so doth that which is seated in the heart, as a sovereign governs all the parts of our lives.
Well, then, here the mark must be fixed. There is no man so good and spiritual that hath not something in him bad and carnal, nor so fully addicted to God but the creature or some inferior good hath an interest in his heart. On the other side, there is no man so addicted to worldly and sensual lusts, that God hath no manner of interest in him at all. Carnal men, if they have not renounced all conscience, and outgrown the heart of a man, have some good [in terms of morals, not true spiritual goodness, which only comes from God] thing in them, and a renewed man hath much of the old tang yet left.
When, then, is a man sincere? Why, when grace gets the upper hand; not for a fit, but habitually. When the soul is more for God than against him; more against sin than for it; more for obeying, loving, serving, and pleasing God, than for gratifying and pleasing the flesh, and your inclination and love to the ways of God is greater than your dislike.
But on the other side, they that love any inferior thing above God are not sincere and upright with him; as those that love pleasure more than God: 2 Tim. iii. 4; John xii. 43, ‘They loved the praise of men more than the praise of God;’ and the profits of the world more than grace: Luke xii. 21, ‘So is he that layeth up treasure for himself, and is not rich towards God.’
These are the great sins which do more directly fight against the sovereignty of God, Therefore these we must look after to see whether they get ground or strength, yea or no, and encroach upon Christ’s interest in our hearts, or the interest of Christ gets ground upon them.
May God grant us so great a love for Him that nothing else in this world matters. May He be everything to us, and may He remove all things from our hearts that are not Him.
