Store 2.0
Raid nach Booten öffnen und mount
Um Auto-Assembly beim Booten zu verhindern muss die Config-Datei /etc/mdadm.conf leer (oder zumindest komplett auskommentiert sein) und "MDADM_SCAN=no" in /etc/sysconfig/mdadm
1.) Checken ob alle Platten da sind:
/root/bin/diskserial_sort2.sh
Müssen im Moment 17 Platten sein.
2.) Raids suchen und assemblen (kein Autostart):
mdadm --assemble --scan
3.) Cryptsetup:
cryptsetup luksOpen /dev/md125 cr_md125
4.) Mounten:
mount /dev/mapper/cr_md125 /data
Schliessen wäre:
cryptsetup luksClose cr_md125
Festplatten-Layout
Serial auslesen mit:
udevadm info --query=all --name=/dev/sdi | grep ID_SERIAL_SHORT
Serial Systemplatte 160GB:
JC0150HT0J7TPC
Serials der Datenplatten
00 : JC0150HT0J7TPC (160GB System)
geht
01 : 02 : 03 : MJ1311YNG3SSLA (3TB) 04 : 05 : MJ1311YNG3NZ3A (3TB)
geht jetzt auch, Molex-Kontakt Problem behoben
06 : 07 : MJ1311YNG4J48A (3TB) 08 : 09 : MJ1311YNG3UUPA (3TB) 10 :
geht
11 : MJ1311YNG3SAMA (3TB) 12 : 13 : MJ1311YNG3SU1A (3TB) 14 : 15 :
geht
16 : 17 : 18 : MJ1311YNG3Y4SA (3TB) 19 : 20 : MJ1311YNG3Y5RA (3TB)
geht jetzt auch, Molex-Kontakt Problem behoben
21 : 22 : 23 : 24 : MJ1311YNG25Z6A (3TB) 25 :
geht
26 : MJ1311YNG3RM5A (3TB) 27 : 28 : MJ1311YNG3NT5A (3TB) 29 : 30 :
geht
31 : 32 : 33 : MJ1311YNG3WZVA (3TB) 34 : 35 : MJ1311YNG3SYKA (3TB)
geht
36 : 37 : 38 : 39 : MJ1311YNG3RZTA (3TB) 40 :
geht
41 : MJ1311YNG3LTRA (3TB) 42 : 43 : MJ1311YNG38VGA (3TB) 44 : 45 :
TOTAL ACTIVE: 17 (16 + eine Systemplatte) von 46 möglichen
Raid Baubefehl
im Screen mdadm
mdadm --create /dev/md125 --chunk=64 --level=raid6 --layout=ls --raid-devices=15 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 /dev/sdn1 /dev/sdo1 /dev/sdp1
Raid-Baustatus
cat /proc/mdstat
automatisch jede Sekunde aktualisiert
watch -n 1 cat /proc/mdstat
Verschlüssel von Hand (ohne Yast2)
Verschlüsseln:
cryptsetup -v --key-size 256 luksFormat /dev/md125
Mit speziellen Paramtern für Hardware-Verschlüsselung:
cryptsetup -v luksFormat --cipher aes-cbc-essiv:sha256 --key-size 256 /dev/md125
Öffnen:
cryptsetup luksOpen /dev/md125 cr_md125
Filesystem drauf:
mkfs.xfs /dev/mapper/cr_md125
store2:~ # mkfs.xfs /dev/mapper/cr_md125
meta-data=/dev/mapper/cr_md125 isize=256 agcount=36, agsize=268435424 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=4096 blocks=9523357168, imaxpct=5
= sunit=16 swidth=208 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=521728, version=2
= sectsz=512 sunit=16 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Status:
cryptsetup luksDump /dev/md125
Samba-Freigabe
There's a bug in Samba in openSuse 11.4. Here's the workaround:
go to Yast --> AppArmor --> Control Panel (on) --> Configure Profile Modes --> usr.sbin.smbd = complain go to Yast --> system --> runlevels --> smb=on + nmb=on reboot
Direkte Netwerkverbindung Store1 <-> Store 2.0
du kannst auch mal schauen was in /etc/udev/rules.d/70-persistent-net... (oder wie auch immer die date heißt) steht.
da wird die mac einer bestimmten netzwerkadresse (eth0, eth1, ...) zugewiesen.
die datei kannst du auch löschen oder verschieben - wird beim neustart neu angelegt.
da kommt machmal was durcheinander - bei 'nem kernelupdate oder bios-update.
GEHT ! unterschiedliche subnets (192.168.2.100 und 192.168.2.102)
Fast-Copy
1.) Empfänger (Store2.0)
cd <Zielverzeichnis> netcat -l -p 4323 | gunzip | cpio -i -d -m
2.) Sender (Store)
cd <Quellverzeichnis> find . -type f | cpio -p | gzip -1 | netcat 192.168.2.102 4323
1.) Empfänger (Store2.0)
socat tcp4-listen:4323 stdout | tar xvpf - /data/eBooks
2.) Sender (Store)
tar cvf - /data/eBooks | socat stdin tcp4:192.168.2.102:4323
Test mit Fortschrittsanzeige bei bekannter Datengröße:
1.) Empfänger (Store2.0)
cd <Zielverzeichnis> socat tcp4-listen:4323 stdout | pv -s 93G | tar xvpf -
2.) Sender (Store)
cd <Quellverzeichnis> tar cvf - * | pv -s 93G | socat stdin tcp4:192.168.2.102:4323
dd if=/dev/sdl | bar -s 1.5T | dd of=/dev/sdw
Crashes
ArchLinux 2011-09-09
Sep 9 18:20:04 localhost kernel: [156439.479947] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Sep 9 18:20:04 localhost kernel: [156439.480035] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Sep 9 18:20:04 localhost kernel: [156439.486612] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Sep 9 18:20:04 localhost kernel: [156439.503656] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Sep 9 18:20:04 localhost kernel: [156439.504562] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO Sep 9 18:34:11 localhost -- MARK -- Sep 9 18:42:42 localhost kernel: [157797.911330] r8169: eth0: link up Sep 9 18:54:11 localhost -- MARK -- Sep 9 19:14:11 localhost -- MARK -- Sep 9 19:34:11 localhost -- MARK -- Sep 9 19:54:11 localhost -- MARK -- Sep 9 20:14:11 localhost -- MARK -- Sep 9 20:27:32 localhost kernel: [164086.971566] r8169: eth0: link up Sep 9 20:27:42 localhost kernel: [164097.580071] r8169: eth0: link up Sep 9 20:27:50 localhost kernel: [164105.391755] r8169: eth0: link up Sep 9 20:27:51 localhost kernel: [164106.272019] r8169: eth0: link up Sep 9 20:28:12 localhost kernel: [164127.150062] r8169: eth0: link up Sep 9 20:28:22 localhost kernel: [164137.941304] r8169: eth0: link up Sep 9 20:28:33 localhost kernel: [164148.890097] r8169: eth0: link up Sep 9 20:28:38 localhost kernel: [164153.080536] r8169: eth0: link up Sep 9 20:28:58 localhost kernel: [164173.790064] r8169: eth0: link up Sep 9 20:42:19 localhost kernel: [ 0.000000] Initializing cgroup subsys cpuset Sep 9 20:42:19 localhost kernel: [ 0.000000] Initializing cgroup subsys cpu Sep 9 20:42:19 localhost kernel: [ 0.000000] Linux version 2.6.32-lts (tobias@T-POWA-LX) (gcc version 4.6.1 20110819 (prerelease) (GCC) ) #1 SMP Tue Aug 30 08:59:44 CEST 2011 Sep 9 20:42:19 localhost kernel: [ 0.000000] Command line: root=/dev/disk/by-uuid/ba47ea9a-c24c-4dc6-a9a2-ca3b442bdbfc ro vga=0x31B Sep 9 20:42:19 localhost kernel: [ 0.000000] KERNEL supported cpus: Sep 9 20:42:19 localhost kernel: [ 0.000000] Intel GenuineIntel Sep 9 20:42:19 localhost kernel: [ 0.000000] AMD AuthenticAMD Sep 9 20:42:19 localhost kernel: [ 0.000000] Centaur CentaurHauls
OpenSuse 2011-09-26
Sep 26 23:15:59 store2 su: (to nobody) root on none Sep 26 23:17:17 su: last message repeated 2 times Sep 26 23:25:23 store2 smartd[4617]: Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 166 to 162 Sep 26 23:25:26 store2 smartd[4617]: Device: /dev/sde [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 171 to 166 Sep 26 23:25:29 store2 smartd[4617]: Device: /dev/sdg [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 171 to 166 Sep 26 23:25:36 store2 smartd[4617]: Device: /dev/sdk [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 171 to 166 Sep 26 23:25:37 store2 smartd[4617]: Device: /dev/sdl [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 181 to 187 Sep 26 23:55:22 store2 smartd[4617]: Device: /dev/sdb [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 96 Sep 26 23:55:23 store2 smartd[4617]: Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 162 to 166 Sep 26 23:55:26 store2 smartd[4617]: Device: /dev/sde [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 99 Sep 26 23:55:26 store2 smartd[4617]: Device: /dev/sde [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 166 to 171 Sep 26 23:55:29 store2 smartd[4617]: Device: /dev/sdg [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 166 to 171 Sep 26 23:55:32 store2 smartd[4617]: Device: /dev/sdi [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 98 Sep 27 00:55:26 store2 kernel: imklog 5.6.5, log source = /proc/kmsg started.
OpenSuse2011-09-27
Sep 27 16:35:17 store2 smbd[29588]: [2011/09/27 16:35:17.391212, 0] param/loadparm.c:8445(check_usershare_stat) Sep 27 16:35:17 store2 smbd[29588]: check_usershare_stat: file /var/lib/samba/usershares/ owned by uid 0 is not a regular file Sep 27 16:44:06 store2 smbd[29163]: [2011/09/27 16:44:06.795153, 0] lib/util_sock.c:474(read_fd_with_timeout) Sep 27 16:44:06 store2 smbd[29163]: [2011/09/27 16:44:06.795341, 0] lib/util_sock.c:1441(get_peer_addr_internal) Sep 27 16:44:06 store2 smbd[29597]: [2011/09/27 16:44:06.795323, 0] lib/util_sock.c:474(read_fd_with_timeout) Sep 27 16:44:06 store2 smbd[29163]: getpeername failed. Error was Der Socket ist nicht verbunden Sep 27 16:44:06 store2 smbd[29592]: [2011/09/27 16:44:06.795368, 0] lib/util_sock.c:474(read_fd_with_timeout) Sep 27 16:44:06 store2 smbd[29163]: read_fd_with_timeout: client 0.0.0.0 read error = Die Verbindung wurde vom Kommunikationspartner zurückgesetzt. Sep 27 16:44:06 store2 smbd[29597]: [2011/09/27 16:44:06.795422, 0] lib/util_sock.c:1441(get_peer_addr_internal) Sep 27 16:44:06 store2 smbd[29597]: getpeername failed. Error was Der Socket ist nicht verbunden Sep 27 16:44:06 store2 smbd[29597]: read_fd_with_timeout: client 0.0.0.0 read error = Die Verbindung wurde vom Kommunikationspartner zurückgesetzt. Sep 27 16:44:06 store2 smbd[29592]: [2011/09/27 16:44:06.795468, 0] lib/util_sock.c:1441(get_peer_addr_internal) Sep 27 16:44:06 store2 smbd[29592]: getpeername failed. Error was Der Socket ist nicht verbunden Sep 27 16:44:06 store2 smbd[29592]: read_fd_with_timeout: client 0.0.0.0 read error = Die Verbindung wurde vom Kommunikationspartner zurückgesetzt. Sep 27 16:45:42 store2 smbd[29585]: [2011/09/27 16:45:42.499038, 0] lib/util_sock.c:474(read_fd_with_timeout) Sep 27 16:45:42 store2 smbd[29593]: [2011/09/27 16:45:42.499082, 0] lib/util_sock.c:474(read_fd_with_timeout) Sep 27 16:45:42 store2 smbd[29593]: [2011/09/27 16:45:42.499174, 0] lib/util_sock.c:1441(get_peer_addr_internal) Sep 27 16:45:42 store2 smbd[29585]: [2011/09/27 16:45:42.499174, 0] lib/util_sock.c:1441(get_peer_addr_internal) Sep 27 16:45:42 store2 smbd[29593]: getpeername failed. Error was Der Socket ist nicht verbunden Sep 27 16:45:42 store2 smbd[29585]: getpeername failed. Error was Der Socket ist nicht verbunden Sep 27 16:45:42 store2 smbd[29593]: read_fd_with_timeout: client 0.0.0.0 read error = Die Verbindung wurde vom Kommunikationspartner zurückgesetzt. Sep 27 16:45:42 store2 smbd[29585]: read_fd_with_timeout: client 0.0.0.0 read error = Die Verbindung wurde vom Kommunikationspartner zurückgesetzt. Sep 27 19:35:14 store2 kernel: imklog 5.6.5, log source = /proc/kmsg started.
OpenSuse 2011-09-29
während kräftigem Copyjob von Store
Sep 29 23:16:19 su: last message repeated 2 times Sep 29 23:28:41 store2 smartd[4624]: Device: /dev/sdb [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 157 to 162 Sep 29 23:28:44 store2 smartd[4624]: Device: /dev/sdd [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 153 to 157 Sep 29 23:28:49 store2 smartd[4624]: Device: /dev/sdh [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 99 Sep 29 23:28:53 store2 smartd[4624]: Device: /dev/sdk [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 162 to 157 Sep 29 23:28:57 store2 smartd[4624]: Device: /dev/sdo [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 176 to 181 Sep 29 23:58:44 store2 smartd[4624]: Device: /dev/sdd [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 99 Sep 29 23:58:49 store2 smartd[4624]: Device: /dev/sdh [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 99 to 100 Sep 29 23:58:53 store2 smartd[4624]: Device: /dev/sdk [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 157 to 162 Sep 29 23:58:57 store2 smartd[4624]: Device: /dev/sdn [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 187 to 193 Sep 29 23:58:58 store2 smartd[4624]: Device: /dev/sdo [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 181 to 176 Sep 29 23:59:02 store2 smartd[4624]: Device: /dev/sdq [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 176 to 181 Sep 30 00:28:41 store2 smartd[4624]: Device: /dev/sda [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 187 to 193 Sep 30 00:28:43 store2 smartd[4624]: Device: /dev/sdd [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 99 to 100 Sep 30 00:28:49 store2 smartd[4624]: Device: /dev/sdh [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 99 Sep 30 00:28:58 store2 smartd[4624]: Device: /dev/sdo [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 176 to 181 Sep 30 00:58:47 store2 smartd[4624]: Device: /dev/sdf [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 99 Sep 30 00:58:49 store2 smartd[4624]: Device: /dev/sdh [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 99 to 100 Sep 30 00:58:59 store2 smartd[4624]: Device: /dev/sdp [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 99 Sep 30 01:28:47 store2 smartd[4624]: Device: /dev/sdf [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 99 to 100 Sep 30 01:28:47 store2 smartd[4624]: Device: /dev/sdf [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 157 to 162 Sep 30 01:28:50 store2 smartd[4624]: Device: /dev/sdi [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 99 Sep 30 01:58:47 store2 smartd[4624]: Device: /dev/sdf [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 162 to 157 Sep 30 01:59:00 store2 smartd[4624]: Device: /dev/sdp [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 99 to 100 Sep 30 02:28:45 store2 smartd[4624]: Device: /dev/sdd [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 99 Sep 30 02:28:46 store2 smartd[4624]: Device: /dev/sde [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 99 Sep 30 02:28:46 store2 smartd[4624]: Device: /dev/sde [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 157 to 162 Sep 30 02:28:48 store2 smartd[4624]: Device: /dev/sdf [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 99 Sep 30 02:28:52 store2 smartd[4624]: Device: /dev/sdi [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 99 to 100 Sep 30 02:58:45 store2 smartd[4624]: Device: /dev/sdd [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 99 to 100 Sep 30 02:58:46 store2 smartd[4624]: Device: /dev/sde [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 99 to 100 Sep 30 02:58:46 store2 smartd[4624]: Device: /dev/sde [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 162 to 157 Sep 30 02:58:47 store2 smartd[4624]: Device: /dev/sdf [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 99 to 100 Sep 30 02:58:49 store2 smartd[4624]: Device: /dev/sdh [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 100 to 99 Sep 30 09:39:22 store2 kernel: imklog 5.6.5, log source = /proc/kmsg started.
What you are seeing are the Normalized Attribute values changing.
For example when the Raw_Read_Error_Rate changed from 99 to 100, the increase in Normalized value from 99 to 100 means that the disk now thinks it is a bit LESS likely to fail than before, because this Normalized value is moving further above the (low) Threshold value.
ArchLinux 2011-10-17
Oct 17 21:21:35 localhost smartd[1941]: Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 181 to 176 Oct 17 21:21:37 localhost smartd[1941]: Device: /dev/sde [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 181 to 176 Oct 17 21:21:45 localhost smartd[1941]: Device: /dev/sdm [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 200 to 206 Oct 17 21:30:03 localhost -- MARK -- Oct 17 21:50:03 localhost -- MARK -- Oct 17 21:51:37 localhost smartd[1941]: Device: /dev/sde [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 176 to 181 Oct 17 21:51:41 localhost smartd[1941]: Device: /dev/sdi [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 181 to 187 Oct 17 22:10:03 localhost -- MARK -- Oct 17 22:21:34 localhost smartd[1941]: Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 176 to 181 Oct 17 22:21:47 localhost smartd[1941]: Device: /dev/sdo [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 206 to 214 Oct 17 22:21:49 localhost smartd[1941]: Device: /dev/sdq [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 206 to 214 Oct 17 22:30:03 localhost -- MARK -- Oct 17 22:50:03 localhost -- MARK -- Oct 17 23:11:18 localhost kernel: [ 0.000000] Initializing cgroup subsys cpuset Oct 17 23:11:18 localhost kernel: [ 0.000000] Initializing cgroup subsys cpu
ArchLinux 2011-11-06
Oct 17 21:21:35 localhost smartd[1941]: Device: /dev/sdc [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 181 to 176 Oct 17 21:21:37 localhost smartd[1941]: Device: /dev/sde [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 181 to 176 Oct 17 21:21:45 localhost smartd[1941]: Device: