Despite always being an important town in the county, the ongoing changes didn’t prevent the conservation of many elements from the medieval town, beginning by the urban layout of the walled town.
In the years 1055 and 1056 the castle was documented as the northern border of the castles of Mur and Llimiana. In the year 1064, Count Ramon VI of Pallars Jussà pledged to give the castle to his cousin Artau I of Pallars Sobirà. In 1079, Ramon VI gave the castle in fief to his castellan Ginaman Hug. In the following years a complicated feudo-vasallatic hierarchy was established, which could be an indication of the importance of the castle in the County, derived from its strategic location and the extent of its jurisdictional territory.
We know that, in the 13th century, among the Talran castellans were members of the Mur and Vilamur families. Arcard of Talarn paid homage to Pere III for the castle of Talarn, in the year 1338. Throughout the first half of the 14th century, the monarchs, who were in need of large sums of money, had repeatedly pawned the castles of Talarn and Orrit. In 1357, the king sold the castles of Talarn and Orrit (with terrains) in allod to the count of Pallars, Arnau Roger, for 71,000 sous. Thanks to royal privilege, from the year 1455 Talarn was the capital of the sotsvegueria of El Pallars and residence of the sotsveguer, a position that going further could only be exercised by citizens of the town. Due to its strategic and political importance, in 1476 it was taken by the mercenary companies of Lluís de Mudarra, and in 1562 by the Huguenots, commanded by the count of Foix. In 1716 Philip V made Talarn the head of corregimiento (es. country subdivision for royal administrative purposes), however in 1834 it lost this position in favor of Tremp.
The Talarn castle was located on the highest and eastern part of the enclosed village. Some remains of it can still be found there today, among which the Prison Tower and the Pla del Castell (an old courtyard). The tower has a 2.7 m diameter and a height of 6 m, with two reliefs at 2.9 and 3.9 m from the ground, and adjacent walls thicknesses of 180 cm. This part can be dated back to the 11th century. Nearby the tower other remains of the castle can be found, including those of a primitive church, hidden among the houses that occupy the space were the castle used to be.
The village of Talarn grew at the foot of the castle. Most likely it started as a castle village, which soon grew, extending the walls of the castle to form an enclosure that embraced the medieval town. Over the years the old medieval houses were modernized. Especially in the times of the towns splendor, when it gained importance at the county level and the main Pallarsian families lived and built their manor house there.
On the eastern side of the town there is the Portal and the Tower of Soldevila, barred and made of large well-arranged ashlars. On the west side of the town, there is the Caps Tower, made with smaller and less regular stones, irregularly arranged. Near this tower, to the north, there are the last few remains of an open tower, nearly vanished. Behind the church of Sant Martí, there are the remains of another tower, of which only a fragment of round construction is conserved (see Sant Martí de Talarn). These towers are newer than the castle, and could be dated between the 13th and 14th centuries.
The enclosed town is formed by few still preserved streets, including the main street (the Mig or the Forn Street) located on the north of the church, parallel to it is the Nou (or Pau Coll) Street, which becomes the main road, since it connects the Portal of Soldevila (south-east) to the Portal of the Caps tower, to the north-west of the village. Further to the north is a third street, which runs through the castel and Vilanova (or the Gall) square, to the west of the village. Inside the enclosure there are only these three streets, however they are intertwined with alleyways, which complete the urban plot of Talarn.