Skid loader starter?

It is a new holland with a ford 4 cylynder at least 15 - 20 years old im am not sure cause it is really beat up. changed out the starter and put in another i turn the key and it will start but when i let the key go it dies like i shut it off