I'm having some trouble trying to figure this out:

James is going to write an exam
Suppose that if he hadn't passed in previous attempts, each time he attempts the exam, he has probability p to pass . The probability that he passes on his first try is 5 times that of passing on his second try.
1) Find E(X)
2) Find the probability that James takes more than 10 times to finish the exam.
3) From (2), deduce that for such distribution, for any n > 0:
P(X > n) = (1 - p)^n

I believe that I should be using binomial distribution in some way. I'm not exactly sure to find the PMF, which I think is required?

Well Binomial distribution usually has fixed number of trials. Each event is independent and can only have one of two outcomes. Flip a coin 20 times. What is the probability of getting 10 heads, less than 7 heads, etc.

Negative Binomial Distribution is when you repeat the experiment an unknown number of times until you have a fixed number of successes. Or failures depending on your textbook. Take a test, repeat until you pass, flip a coin until you get heads. Or until you get 3 tails. Roll two dice until you get a 7, etc. Like binomial distribution, each trial is independent and can only have one of two outcomes.