29 Jan 2023, 00:18
It's been an hour since things went dark - all at once: not the off, on, off, on, off to stay of a power co-op breaker trying to reset but just instant dark.
The local power co-op's website and their app say power went off at 23:20 but I looked at the clock on the laptop when the lights went out and it was 23:18. All 676 meters fed by the aerial 7400 volt lines that run along the two miles of main road were taken out in one fell swoop. The point of failure is just West of an intersection where the feed for one road (us and 675 other meters) heads West and another feed goes East. No Estimated Restoration Time (ERT) or description of the failure from our co-op either online or via the app - not something they have ever provided.
Twenty minutes later, power is still off so I grabbed a rechargeable head light and went to the basement to get the backup power system going. It's a 12 volt, 420AH AGM battery bank powering a 2000 watt pure sine wave inverter, something I've been exploring and expanding for almost six years as I teach myself about solar power. Battery charging depends on the season and my health - I'm currently recoving from spinal fusion surgery so it's an AC operated power supply that keeps the battery bank charged without needing me to be outside to clean solar panels. My timing was good as the furnace's induction blower started as soon as it had power ;-) I did report the outage using the co-op's app when I got back upstairs. Other than a few solar walkway lights, it's very dark in every direction outside ;-)
I'm using one of the laptop PCs to check power status (the head light is on the sofa and aimed up to have light reflected from the ceiling on the keyboard) and the better half is using a USB powered LED light on a 5000mah power bank to have light to read a magazine by. Without the grid, you use power where and as needed - no point in lighting an entire room when the paper is in your lap.
We're good for about 8 hours (furnace & fridge, maybe freezer) on silent backup power. Other than not watching anything recorded on the DVR, it will be an almost normal night ;-)
However, some folks think you don't need backup power unless there's big storm coming...
From memory, we had almost the identical failure a year or so ago: 676 meters for about two hours with the failure at the same point, and four years ago near that location. Guess they didn't fix things adequately either of those times...
===
Two and a half hours into this undefined outage and with no info from the co-op, I got a Luci Core light (from MPOWERD.com) that hangs over a curtain rod in the dining room with its solar panel facing the sunny window. They spec that light as good for 12 hours on its lowest setting so I should have light as long as I want to stay up and watch status on this outage (actual run time of the Luci was more like 6 hours).
I do have remote access to the laptop monitoring the solar backup system - UltraViewer is a very useful free tool. The battery bank on the backup system is at 12.28 volts and my usual cutoff point for AGM batteries is 50% DOD which will be around 12.1 volts under load. I can hear if the fridge or the furnace is running, so I know whether the battery voltage I'm looking at is near resting (now 12.76 volts, with just couple of LED lights on and a couple of rechargeable portable lights on charge) or the system is actually powering something that uses in the hundreds of watts (460 watts for the furnace blower or 560 watts for the fridge in defrost mode would justify the 12.28 volts). If this lasts more than four hours, I'll run a long extension cord to provide power to the freezer for an hour or so to ensure it stays cold enough. It's not been opened so it should be fine for 8-16 hours (CDC says 24 hours but I'm a pessimist). However, I have a volts, amps, watts, amp hours and watt hours monitor on the battery bank so I can check for the actual used AH and shut the backup system down when it's used 50% of the battery bank's rated AH capacity. That's the guaranteed way to not exceed 50% discharge (or whatever value you use for your battery backup).
Because I can watch the voltage remotely, I'll not make another trip to the basement to check the AH used until the battery voltage is closer to 12.1 volts. Nice to have backup when almost no one else does (there's a house up on a hill that appears to have whole house backup with all the windows lit but I wouldn't want to be the only light in a large dark space). We've had light showing, but most of the lights have moved from room to room and it's been, at most, one or two rooms with light.
My better half was happy with the rechargeable light on top of the fridge - it lighted the interior of the upper cabinets better than the overhead light in the kitchen does when she was emptying the dishwasher and putting things away. There are occasions when my "weird ways" get appreciated ;-)
Researching failures in the area, a similar failure that affected 800+ meters occurred about 4 years ago and a nearly identical failure affecting 676 meters happened about a year and a half ago (May 2021, same failure point and almost 2 hours of outage). Makes you wonder how much shade tree mechanic-ing goes on in that section of the power distribution network...
=====
Four and a half hours into this outage and I've had to do a little work.
1. The UPS units for the home network don't last more than 3 hours. I ran a 50 foot extension cord to that part of the basement and used a triple cube tap to be able to plug in two UPS units and the 24 port network switch - that device's UPS died during this event (the battery in that UPS is 4 years old and that's about the limit for AGM batteries in most UPS units). Not an expensive UPS unit as all I have in it is the price of the battery (about $20) because the UPS itself was a freebie on Craig's List. I replaced that UPS with a smaller one that I think has one of the 12 volt, 3.8AH LiFePO4 battery packs I built for the UPS units with small loads that should run a long time (network router and switches, laptop monitoring the solar backup system, etc.).
2. After four hours, I plugged the freezer in long enough for it to run and cycle off once. That says it's cold enough for now. If this outage lasts much longer, I may put the freezer on a digital timer that lets it run one hour out of five.
3. The laptop I've been using needed to be charged so it's on the kitchen counter and 30 minutes or so from being fully charged. Meanwhile, I'm using an ancient Dell D630 laptop running XP. It also needed to be charged so I unreeled the reel extension cord with four outlets, plugged the reel into the last free outlet with backup power in the kitchen and brought power to the family room. I'm using two of the reel's outlets to charge two other UPS units, just in case they are needed later. The batteries in one of those UPS units seem to have reached the "charged" state as it did several seconds of relay clicking as it switched to battery power to see if that would work - I think that one is charged.
When the laptop is charged, I may move the reel outlets over to the pure sine wave UPS that serves the A/V cabinet and let that one charge so I might be able to watch a bit of TV while I wait. I need to sit and wait for a while as I've been on my feet and up and down stairs far too much this night :-( While I have power near the sofa, I might plug in two heating pads - one for my back and one for my neck. Getting certain things unplugged so they can be connected to backup power at the end of an extension cord can sometimes require a lot of bending and twisting - things that are part of the Thou Shalt Not list from my back surgery. I'm a tablet and a half into the Rx pain meds (I consider two tablets the max for a single dose but I try to stay with the minimum dose that lets me function or sleep) - hopefully I'll be testing the level of sleep the current dose can provide in a few hours - but I'm not optimistic :-(
====
At 6 hours in, this outage still has NO additional information posted by the power co-op - as is typical of this co-op.
The freezer ran longer than I anticipated so I need to charge the batteries a bit to keep up with the fridge for a few more hours - no generator in use here before daylight because I don't want to wake someone who doesn't yet know power has been off for hours and who might want me to "share" my power. I set the house thermostat down two degrees so the furnace shouldn't kick back in soon and I'll turn on the gas logs in the family room so there will be more warm air available when the furnace does run again and the furnace should run for less time with a roomful of warmer air available. How will I charge the batteries when there is no power, the sun is several hours away and I never run a generator in the dark? I pressed two of the newest members of the battery stable into service. Two of the 12 volt, 10AH LiFePO4 batteries were wired in series and connected to the "solar" input on one of the MPPT charge controllers that's connected to the battery bank. The charge controller doesn't care where the DC voltage on the "solar" input comes from, just that there is power there. That controller is trucking along at 398 watts output to the battery bank (13.x volts at 29 amps). The charging process won't last long as there are only 260 watt hours of power in those batteries (10 amp hours at 26 volts) and, at best, the throughput will be a little more than 90% of that value. That added power will cover another hour of power for the inverter to power the fridge, furnace, etc.). I have another pair of those LiFePO4 batteries that could be used to add roughly that much more power to the battery bank and maybe make it until daylight. Solar panels are NOT a good option until after 10AM so I guess the better half and I will be muscling the 50lbs of inverter gen out of the shed so it can run the essential appliances and charge the batteries. If I would just turn loose of about $2600, I could have a 12 volt, 920AH LiFePO4 battery bank that would run things for more than 24 hours without any attention ;-) I'm still waiting for the TV/movie offer on one or more of my published books so I don't think this will be the year for the LiFePO4 upgrade :-(
Regardless of the glitches, the backup system did what was expected - the freezer is on the list after eight hours of outage but I connected it at four hours so more power was used sooner than the spreadsheet specifies.
In checking the battery bank remotely, the voltage is beginning to decrease so I think the first two small LiFePO4 batteries have given their best and it's time to put in the next team of LFP ;-) That should also buy another hour of use so figure two hours to 7:30 and sunrise. If power's not back then, it will be time to run the generator.
====
I also used two 35AH AGM batteries from some previous solar testing to provide additional charge to the battery bank - just adding about an hour's worth of power at a time.
As we approach 8 hours into this outage, the power co-op still hasn't posted any additional info about the failure.
---
However, at one minute before the 8 hour mark, power came back on. Now I get to put ALL those batteries on charge...
At least I had or found options that kept the fridge and furnace running on silent backup until commercial power came back ;-)
Epilog:
Twenty four hours after power was restored, almost all signs of the outage are gone from the basement. The six batteries pressed into service for small, partial recharges of the primary battery bank are now fully charged and waiting to be used as needed. The four power supplies used to charge those batteries via appropriate charge controllers have been put away and the spare charge controllers that got their semi-regular load test are also put away for future need. Yes, all the spares worked so I'm confident they will work when/if they are again needed. A few power cables are on the workbench and waiting to be hung back on their storage hooks - something I would have done earlier but I'm a tablet and a half into the Rx pain meds for my back and those cables are not valid reason for another trip down and up the stairs. I obviously did too much tonight with the extra trips down and up the basement stairs. I see the surgeon tomorrow and I'll ask "How long until I'm back to normal?"
Our power co-op never announced what the problem was or why it took eight hours to fix it. Not the most useful status reporting. Perhaps I was spoiled by having Georgia Power as our power provider for about 30 years; their outage reports always include cause (if/when known) and an ERT (Estimated Restoration Time).
One rechargeable light is still on charge on the kitchen counter but that should be done soon and can be put back in the "In the dark" box that lives behind a chair in the family room. That box has USB lights and 5 volt power banks from 2000 to 10000mah and assorted small rechargeable lights (USB and handcrank). The room does have light when power goes off as the TV has a USB-powered LED strip that lights the wall behind the TV and the pure sine wave UPS for the TV and DVR ensures the room does not go totally dark when grid power is lost so it's a good place for the "In the dark" box.
---
It's been an hour since things went dark - all at once: not the off, on, off, on, off to stay of a power co-op breaker trying to reset but just instant dark.
The local power co-op's website and their app say power went off at 23:20 but I looked at the clock on the laptop when the lights went out and it was 23:18. All 676 meters fed by the aerial 7400 volt lines that run along the two miles of main road were taken out in one fell swoop. The point of failure is just West of an intersection where the feed for one road (us and 675 other meters) heads West and another feed goes East. No Estimated Restoration Time (ERT) or description of the failure from our co-op either online or via the app - not something they have ever provided.
Twenty minutes later, power is still off so I grabbed a rechargeable head light and went to the basement to get the backup power system going. It's a 12 volt, 420AH AGM battery bank powering a 2000 watt pure sine wave inverter, something I've been exploring and expanding for almost six years as I teach myself about solar power. Battery charging depends on the season and my health - I'm currently recoving from spinal fusion surgery so it's an AC operated power supply that keeps the battery bank charged without needing me to be outside to clean solar panels. My timing was good as the furnace's induction blower started as soon as it had power ;-) I did report the outage using the co-op's app when I got back upstairs. Other than a few solar walkway lights, it's very dark in every direction outside ;-)
I'm using one of the laptop PCs to check power status (the head light is on the sofa and aimed up to have light reflected from the ceiling on the keyboard) and the better half is using a USB powered LED light on a 5000mah power bank to have light to read a magazine by. Without the grid, you use power where and as needed - no point in lighting an entire room when the paper is in your lap.
We're good for about 8 hours (furnace & fridge, maybe freezer) on silent backup power. Other than not watching anything recorded on the DVR, it will be an almost normal night ;-)
However, some folks think you don't need backup power unless there's big storm coming...
From memory, we had almost the identical failure a year or so ago: 676 meters for about two hours with the failure at the same point, and four years ago near that location. Guess they didn't fix things adequately either of those times...
===
Two and a half hours into this undefined outage and with no info from the co-op, I got a Luci Core light (from MPOWERD.com) that hangs over a curtain rod in the dining room with its solar panel facing the sunny window. They spec that light as good for 12 hours on its lowest setting so I should have light as long as I want to stay up and watch status on this outage (actual run time of the Luci was more like 6 hours).
I do have remote access to the laptop monitoring the solar backup system - UltraViewer is a very useful free tool. The battery bank on the backup system is at 12.28 volts and my usual cutoff point for AGM batteries is 50% DOD which will be around 12.1 volts under load. I can hear if the fridge or the furnace is running, so I know whether the battery voltage I'm looking at is near resting (now 12.76 volts, with just couple of LED lights on and a couple of rechargeable portable lights on charge) or the system is actually powering something that uses in the hundreds of watts (460 watts for the furnace blower or 560 watts for the fridge in defrost mode would justify the 12.28 volts). If this lasts more than four hours, I'll run a long extension cord to provide power to the freezer for an hour or so to ensure it stays cold enough. It's not been opened so it should be fine for 8-16 hours (CDC says 24 hours but I'm a pessimist). However, I have a volts, amps, watts, amp hours and watt hours monitor on the battery bank so I can check for the actual used AH and shut the backup system down when it's used 50% of the battery bank's rated AH capacity. That's the guaranteed way to not exceed 50% discharge (or whatever value you use for your battery backup).
Because I can watch the voltage remotely, I'll not make another trip to the basement to check the AH used until the battery voltage is closer to 12.1 volts. Nice to have backup when almost no one else does (there's a house up on a hill that appears to have whole house backup with all the windows lit but I wouldn't want to be the only light in a large dark space). We've had light showing, but most of the lights have moved from room to room and it's been, at most, one or two rooms with light.
My better half was happy with the rechargeable light on top of the fridge - it lighted the interior of the upper cabinets better than the overhead light in the kitchen does when she was emptying the dishwasher and putting things away. There are occasions when my "weird ways" get appreciated ;-)
Researching failures in the area, a similar failure that affected 800+ meters occurred about 4 years ago and a nearly identical failure affecting 676 meters happened about a year and a half ago (May 2021, same failure point and almost 2 hours of outage). Makes you wonder how much shade tree mechanic-ing goes on in that section of the power distribution network...
=====
Four and a half hours into this outage and I've had to do a little work.
1. The UPS units for the home network don't last more than 3 hours. I ran a 50 foot extension cord to that part of the basement and used a triple cube tap to be able to plug in two UPS units and the 24 port network switch - that device's UPS died during this event (the battery in that UPS is 4 years old and that's about the limit for AGM batteries in most UPS units). Not an expensive UPS unit as all I have in it is the price of the battery (about $20) because the UPS itself was a freebie on Craig's List. I replaced that UPS with a smaller one that I think has one of the 12 volt, 3.8AH LiFePO4 battery packs I built for the UPS units with small loads that should run a long time (network router and switches, laptop monitoring the solar backup system, etc.).
2. After four hours, I plugged the freezer in long enough for it to run and cycle off once. That says it's cold enough for now. If this outage lasts much longer, I may put the freezer on a digital timer that lets it run one hour out of five.
3. The laptop I've been using needed to be charged so it's on the kitchen counter and 30 minutes or so from being fully charged. Meanwhile, I'm using an ancient Dell D630 laptop running XP. It also needed to be charged so I unreeled the reel extension cord with four outlets, plugged the reel into the last free outlet with backup power in the kitchen and brought power to the family room. I'm using two of the reel's outlets to charge two other UPS units, just in case they are needed later. The batteries in one of those UPS units seem to have reached the "charged" state as it did several seconds of relay clicking as it switched to battery power to see if that would work - I think that one is charged.
When the laptop is charged, I may move the reel outlets over to the pure sine wave UPS that serves the A/V cabinet and let that one charge so I might be able to watch a bit of TV while I wait. I need to sit and wait for a while as I've been on my feet and up and down stairs far too much this night :-( While I have power near the sofa, I might plug in two heating pads - one for my back and one for my neck. Getting certain things unplugged so they can be connected to backup power at the end of an extension cord can sometimes require a lot of bending and twisting - things that are part of the Thou Shalt Not list from my back surgery. I'm a tablet and a half into the Rx pain meds (I consider two tablets the max for a single dose but I try to stay with the minimum dose that lets me function or sleep) - hopefully I'll be testing the level of sleep the current dose can provide in a few hours - but I'm not optimistic :-(
====
At 6 hours in, this outage still has NO additional information posted by the power co-op - as is typical of this co-op.
The freezer ran longer than I anticipated so I need to charge the batteries a bit to keep up with the fridge for a few more hours - no generator in use here before daylight because I don't want to wake someone who doesn't yet know power has been off for hours and who might want me to "share" my power. I set the house thermostat down two degrees so the furnace shouldn't kick back in soon and I'll turn on the gas logs in the family room so there will be more warm air available when the furnace does run again and the furnace should run for less time with a roomful of warmer air available. How will I charge the batteries when there is no power, the sun is several hours away and I never run a generator in the dark? I pressed two of the newest members of the battery stable into service. Two of the 12 volt, 10AH LiFePO4 batteries were wired in series and connected to the "solar" input on one of the MPPT charge controllers that's connected to the battery bank. The charge controller doesn't care where the DC voltage on the "solar" input comes from, just that there is power there. That controller is trucking along at 398 watts output to the battery bank (13.x volts at 29 amps). The charging process won't last long as there are only 260 watt hours of power in those batteries (10 amp hours at 26 volts) and, at best, the throughput will be a little more than 90% of that value. That added power will cover another hour of power for the inverter to power the fridge, furnace, etc.). I have another pair of those LiFePO4 batteries that could be used to add roughly that much more power to the battery bank and maybe make it until daylight. Solar panels are NOT a good option until after 10AM so I guess the better half and I will be muscling the 50lbs of inverter gen out of the shed so it can run the essential appliances and charge the batteries. If I would just turn loose of about $2600, I could have a 12 volt, 920AH LiFePO4 battery bank that would run things for more than 24 hours without any attention ;-) I'm still waiting for the TV/movie offer on one or more of my published books so I don't think this will be the year for the LiFePO4 upgrade :-(
Regardless of the glitches, the backup system did what was expected - the freezer is on the list after eight hours of outage but I connected it at four hours so more power was used sooner than the spreadsheet specifies.
In checking the battery bank remotely, the voltage is beginning to decrease so I think the first two small LiFePO4 batteries have given their best and it's time to put in the next team of LFP ;-) That should also buy another hour of use so figure two hours to 7:30 and sunrise. If power's not back then, it will be time to run the generator.
====
I also used two 35AH AGM batteries from some previous solar testing to provide additional charge to the battery bank - just adding about an hour's worth of power at a time.
As we approach 8 hours into this outage, the power co-op still hasn't posted any additional info about the failure.
---
However, at one minute before the 8 hour mark, power came back on. Now I get to put ALL those batteries on charge...
At least I had or found options that kept the fridge and furnace running on silent backup until commercial power came back ;-)
Epilog:
Twenty four hours after power was restored, almost all signs of the outage are gone from the basement. The six batteries pressed into service for small, partial recharges of the primary battery bank are now fully charged and waiting to be used as needed. The four power supplies used to charge those batteries via appropriate charge controllers have been put away and the spare charge controllers that got their semi-regular load test are also put away for future need. Yes, all the spares worked so I'm confident they will work when/if they are again needed. A few power cables are on the workbench and waiting to be hung back on their storage hooks - something I would have done earlier but I'm a tablet and a half into the Rx pain meds for my back and those cables are not valid reason for another trip down and up the stairs. I obviously did too much tonight with the extra trips down and up the basement stairs. I see the surgeon tomorrow and I'll ask "How long until I'm back to normal?"
Our power co-op never announced what the problem was or why it took eight hours to fix it. Not the most useful status reporting. Perhaps I was spoiled by having Georgia Power as our power provider for about 30 years; their outage reports always include cause (if/when known) and an ERT (Estimated Restoration Time).
One rechargeable light is still on charge on the kitchen counter but that should be done soon and can be put back in the "In the dark" box that lives behind a chair in the family room. That box has USB lights and 5 volt power banks from 2000 to 10000mah and assorted small rechargeable lights (USB and handcrank). The room does have light when power goes off as the TV has a USB-powered LED strip that lights the wall behind the TV and the pure sine wave UPS for the TV and DVR ensures the room does not go totally dark when grid power is lost so it's a good place for the "In the dark" box.
---