you handle the sentiment well - I like the simplicity - truncation at end of second line is a bit awkward and jarring - try running the line over into the next:
...quick to greet you. Bow you're head
nix the "the" in the last line (as in "the he will make you") and clean up the punctuation
it could go something like this:
The taste of passion is bittersweet. In the morning, guilt is quick to greet you. Bow you're head low towards the Lord and he will make you feel complete.
or you could find another rhyme/slant rhyme for greet
or you could kill bittersweet and find a different, less well-worn word - and another rhyming word to end 2nd line...
lots of ways to clean it up
don't jazz it up much, though - this is a heartfelt religious sentiment and works well in simple form
unsure how much critique you really wanted - I've tried to be blunt and honest in these comments, showing you the respect due a fellow poet seeking c/c - as for the star ratings, ignore them, I'm still new here and would have to see the scale played out in more critiques to make them accurately reflect my sentiments - what I'm saying with the star ratings is vision and originality are not the strong suit here but for this type of pedagogical, religiously inspired poem, this is not surprising - would have given one more star for technique but for second line ending and the added "the"
this could touch some people when cleaned up - make it as good as you can - make it your own - hope something here helps you do that
The first two lines leave the piece open to wonderful interpretations of why the morning must bring guilt, and what exactly the persona in this piece has been up to, in order to deserve the guilt. However, the last line throws off the balance of the poem. The pattern of syllables doesn't match with the rest of the stanza, and the increased length of the third line increases the fractured appearance of a very short last line. This seems to create a stalled and stuttered ending, not quite the sort of tone that complements the idea of forgiveness and the supposed serenity that would come along with it. Very short pieces can be incredibly useful and packing a punch, and really getting a theme across to the reader in a powerful way. But to do this you must really condense the story line and remember that every word counts. My advice would be to run through this again, read it out-loud in order to see if the rhythm works, and also look at each word individually. Make sure that the words you pick all help towards building your story/narrative/idea. A very nice starting point, I look forward to coming across your work again.