Lost and Prison break were similar for me. I loved the first season, liked the second season and then eventually they became unwatchable. I haven't finished either show yet. I'm stopped halfway through season 4 on Prison Break and I stopped with lost once they started with the flashbacks.
It was a fun show, but, even in the first season they had trouble finding enough plot. There were only so many pieces of information that could hide in the tattoos on his body.
Still, it's a show that you look back and you appreciate the chance they took to put that on the air. It was definitely something that came on the air in response to "Lost" showing that viewers will watch a network drama with ongoing and complex storylines.
I also think that they did a decent job with season 2 and 3 after they got out of the prison. When the show was in its first season, half the comments I ever heard about the show was that it would be impossible to continue the show after they got out, but, they managed to do a good job. Adding William Fichter to the cast was a smart move. I think I stopped watching when it came back for the fourth season though.
If this show came on the air today, I'm not sure I would watch it though. I'll watch sitcoms on network television, but, am really reluctant to give any network drama a try unless I hear great reviews on it, and instead, try to stick to the cable shows.