Consistency is the most important thing. Not only at the same time of day but the SAME SCALE. Some people don't think about this, but every scale weighs just a little differently.
I personally weigh in the morning, right after I get out of the shower, naked.
Also keep in mind if you weigh later in the day, anything you've eaten or drunk will add to your weight. A gallon of water weighs 8.4 lbs. So if you drink a 20 oz bottle of water and step on the scale, you're going to weigh 1.4 lbs more because of the weight of the water in your stomach.
Plus, whatever you wear shows up on the scale. So if you weigh on Monday in your
gym clothes - shorts and a tshirt - and then you weight on Tuesday in jeans and a shirt, you're going to weigh more because your clothes weigh more.
The scale weighs EVERYTHING. So keep that in mind when you choose your time to weigh.