Built in 1756, the stone house of Casa Góngora (corner of Avenida Central and Calle 4, tel. 212-0338, 8 a.m.–4 p.m. Mon.–Fri., closed Sat. and Sun., free) is the oldest house in Casco Viejo  and one of the oldest in Panama. It was originally the home of a Spanish pearl merchant. It then became a church and has now been turned into a small, bare-bones museum.
It’s had a rough history—it has been through three fires and the current wooden roof is new. A 20th-century restoration attempt was botched, causing more damage. There isn’t much here, but the staff can give free tours (in Spanish) and there have been noises about making it more of a real museum in the future.
There’s an interesting, comprehensive book on the history of the house and neighborhood (again, in Spanish) that visitors are welcome to thumb through while visiting. It contains rare maps, photos, and illustrations. Ask for it at the office. The museum hosts jazz and folkloric concerts and other cultural events in the tiny main hall on some Friday and Saturday nights, and it occasionally hosts art shows.
Iglesia San Felipe de Neri (corner of Avenida B and Calle 4) dates from 1688 and, though it has also been damaged by fires, is one of the oldest standing structures from the Spanish colonial days. It was recently renovated but wasn’t open when I last tried to check it out.