The Rams played in Los Angeles from 1946-1994(technically in Anaheim a suburb from 80-94) and the Raiders from 1982-1994 both left LA in the exact same year, the Chargers also played in Los Angeles in 1960(in the AFL in the pre merger days, which is now the AFC conference) before moving down the coast to LA.
The big knock against Los Angeles for years was the crappy stadium situation which is one of the reasons that both the Raiders and Rams left in the same year, both teams were also splitting the potential fan base and the Rams for years were a poor side. The NFL twenty years ago wasn't quite as healthy as it is today either. San Diego is only a couple of hours away as well so some fans "adopted" them as their team or they simply started supporting a franchise that had players they liked.
Some football fans in Los Angeles who are passionate supporters of other teams actually don't want a team to relocate, since the local teams broadcasts will override any out of region games and fans will get stuck watching LA or forced to pay big money for NFL viewing packages(I suppose "illegal" viewing would be an option but it's always a bit skecthy)