What test was the p-value associated with? I would say that there IS a fairly strong association. First of all, don't let the .05 rule ruin your day. The .05 is arbitrary. I would consider the .056 very suggestive, especially if your sample size is small.
Second, don't let the OR less than 1 (decreased odds) fool you. A 0.65 is actually pretty strong. This is equivalent to an OR of 1.54 (Increased odds), which is strong.