One of my two sets of data (x) is normally distributed, but the second (y) has some values that could be outliers. When I make a histogram of my y data it looks something like this:

*(If the image does not show in the message post you can see it at this link: http://tinypic.com/r/w14ldc/5)*

When I remove the outliers (to the right) the histogram looks like a normal distribution (the data also meets other tests of a normal distribution).

Here is my conundrum.

There is very, very little difference for r squared and P from the linear regression between leaving the outliers in and taking them out. Linear regression with the outliers left in the data results in an r squared of 0.201 and a P < 0.00001. Linear regression with the outliers removed results in an r squared of 0.198 and still a P <0.00001. The only difference is my resulting y=mx+b equation and the y equation works a lot better when the outliers are left in my analysis.

So given the extremely significant P values and very little r squared difference do I still have to remove the outliers? Or under these circumstances would using linear regression still be appropriate even though my y data does not appear to be from a normal distribution?